Sep 10 20:58:24.152214 ip-10-0-128-196 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 20:58:24.584030 ip-10-0-128-196 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:24.584030 ip-10-0-128-196 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 20:58:24.584030 ip-10-0-128-196 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:24.584030 ip-10-0-128-196 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 20:58:24.584030 ip-10-0-128-196 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:24.584683 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.584050    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 20:58:24.588198 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588178    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:24.588198 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588197    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588201    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588205    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588208    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588211    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588214    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588217    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588222    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588224    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588227    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588229    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588233    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588236    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588239    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588242    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588244    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588247    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588251    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588253    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588256    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:24.588271 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588258    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588261    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588264    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588266    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588269    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588272    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588274    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588278    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588280    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588283    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588286    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588289    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588291    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588294    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588297    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588299    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588303    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588305    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588308    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588311    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:24.588753 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588314    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588316    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588319    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588323    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588325    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588328    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588330    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588333    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588336    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588338    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588341    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588343    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588346    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588348    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588352    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588355    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588358    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588360    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588363    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588366    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:24.589253 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588370    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588372    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588375    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588377    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588380    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588382    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588385    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588388    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588390    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588393    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588396    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588398    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588401    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588403    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588406    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588408    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588411    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588413    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588417    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588421    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:24.589750 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588424    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588427    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588430    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588433    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588435    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588438    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588441    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588444    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588446    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588449    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588451    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588455    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588459    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588461    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588464    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588467    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588471    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.588474    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589206    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:24.590265 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589213    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589216    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589218    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589222    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589225    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589228    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589230    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589233    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589236    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589239    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589243    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589246    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589248    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589251    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589254    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589257    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589259    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589262    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589266    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:24.590757 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589269    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589271    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589274    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589277    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589280    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589283    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589285    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589288    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589292    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589296    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589299    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589302    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589304    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589307    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589309    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589312    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589314    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589317    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589319    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:24.591246 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589324    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589328    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589331    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589334    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589338    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589341    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589344    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589347    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589350    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589352    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589355    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589358    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589361    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589363    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589366    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589369    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589371    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589374    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589377    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589379    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:24.591798 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589381    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589384    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589386    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589389    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589391    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589394    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589396    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589399    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589401    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589404    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589406    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589409    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589412    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589415    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589417    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589419    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589422    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589425    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589427    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589430    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:24.592314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589432    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589435    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589437    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589439    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589443    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589446    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589448    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589451    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589453    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589456    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589459    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589461    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589464    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589466    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589469    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589472    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589475    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589477    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589482    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:24.592807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589485    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589897    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589902    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589905    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589909    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589912    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589914    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589917    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589919    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589922    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589925    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589928    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589931    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589933    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589936    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589938    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589940    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589943    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589946    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589949    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:24.593314 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589951    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589954    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589957    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589959    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589962    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589964    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589967    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589969    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589972    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589974    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589976    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589979    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589981    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589984    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589986    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589989    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589993    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.589998    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590004    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:24.593807 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590010    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590058    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590064    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590068    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590077    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590082    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590086    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590089    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590091    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590094    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590097    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590100    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590102    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590106    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590108    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590111    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590114    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590117    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590119    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:24.594328 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590122    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590125    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590127    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590130    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590133    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590137    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590140    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590146    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590149    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590152    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590155    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590158    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590161    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590164    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590167    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590171    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590174    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590177    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590180    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590182    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:24.594787 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590185    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590187    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590192    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590195    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590197    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590200    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590203    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590206    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590208    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590211    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590214    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590216    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590220    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590223    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590225    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590228    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590230    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590233    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590236    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:24.595287 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590239    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590242    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.590244    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590328    2584 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590366    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590374    2584 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590380    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590386    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590389    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590394    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590399    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590402    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590405    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590409    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590413    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590416    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590419    2584 flags.go:64] FLAG: --cgroup-root=""
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590422    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590425    2584 flags.go:64] FLAG: --client-ca-file=""
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590428    2584 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590432    2584 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590436    2584 flags.go:64] FLAG: --cluster-domain=""
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590439    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590443    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 20:58:24.595777 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590446    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590449    2584 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590453    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590457    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590460    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590463    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590467    2584 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590470    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590473    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590476    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590480    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590484    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590487    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590490    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590494    2584 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590497    2584 flags.go:64] FLAG: --enable-server="true"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590501    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590506    2584 flags.go:64] FLAG: --event-burst="100"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590510    2584 flags.go:64] FLAG: --event-qps="50"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590513    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590516    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590519    2584 flags.go:64] FLAG: --eviction-hard=""
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590523    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590526    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590529    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 20:58:24.596421 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590532    2584 flags.go:64] FLAG: --eviction-soft=""
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590535    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590538    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590541    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590544    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590547    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590550    2584 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590553    2584 flags.go:64] FLAG: --feature-gates=""
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590557    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590560    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590563    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590566    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590569    2584 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590573    2584 flags.go:64] FLAG: --help="false"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590576    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-128-196.ec2.internal"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590579    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590582    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590584    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590589    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590592    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590595    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590598    2584 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590602    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 20:58:24.597059 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590605    2584 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590608    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590611    2584 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590614    2584 flags.go:64] FLAG: --kube-reserved=""
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590617    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590619    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590623    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590626    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590629    2584 flags.go:64] FLAG: --lock-file=""
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590631    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590634    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590638    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590643    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590646    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590649    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590652    2584 flags.go:64] FLAG: --logging-format="text"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590654    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590658    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590660    2584 flags.go:64] FLAG: --manifest-url=""
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590663    2584 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590668    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590671    2584 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590676    2584 flags.go:64] FLAG: --max-pods="110"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590679    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590682    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 20:58:24.597656 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590685    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590688    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590691    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590695    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590698    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590704    2584 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590707    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590710    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590713    2584 flags.go:64] FLAG: --pod-cidr=""
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590716    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590722    2584 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590725    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590728    2584 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590731    2584 flags.go:64] FLAG: --port="10250"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590734    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590738    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-027b26539de780acd"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590741    2584 flags.go:64] FLAG: --qos-reserved=""
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590744    2584 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590746    2584 flags.go:64] FLAG: --register-node="true"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590749    2584 flags.go:64] FLAG: --register-with-taints=""
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590753    2584 flags.go:64] FLAG: --registry-burst="10"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590756    2584 flags.go:64] FLAG: --registry-qps="5"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590759    2584 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590762    2584 flags.go:64] FLAG: --reserved-memory=""
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590766    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 20:58:24.598301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590770    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590773    2584 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590776    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590779    2584 flags.go:64] FLAG: --runonce="false"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590782    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590785    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590788    2584 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590791    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590794    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590797    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590800    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590811    2584 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590814    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590817    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590820    2584 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590823    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590826    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590829    2584 flags.go:64] FLAG: --system-cgroups=""
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590832    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590838    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590841    2584 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590858    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590862    2584 flags.go:64] FLAG: --tls-min-version=""
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590865    2584 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590868    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 20:58:24.599256 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590871    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590873    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590877    2584 flags.go:64] FLAG: --v="2"
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590886    2584 flags.go:64] FLAG: --version="false"
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590890    2584 flags.go:64] FLAG: --vmodule=""
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590894    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.590897    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591018    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591022    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591025    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591027    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591030    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591033    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591036    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591039    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591042    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591045    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591047    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591050    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591053    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591056    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:24.599983 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591058    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591061    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591063    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591066    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591069    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591071    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591074    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591076    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591080    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591082    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591085    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591088    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591091    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591094    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591096    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591099    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591102    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591105    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591107    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591110    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:24.600492 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591113    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591115    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591118    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591120    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591123    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591126    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591128    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591131    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591134    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591137    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591139    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591143    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591145    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591148    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591151    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591153    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591156    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591158    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591161    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591164    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:24.601032 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591166    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591169    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591172    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591174    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591177    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591180    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591182    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591185    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591188    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591190    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591193    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591197    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591201    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591204    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591207    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591210    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591213    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591215    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591218    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:24.601519 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591223    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591226    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591228    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591231    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591234    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591630    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591636    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591638    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591641    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591645    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591648    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591651    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591655    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591658    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591661    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591663    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591666    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591669    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591671    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591674    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:24.601997 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591677    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591679    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591682    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591685    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591687    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.591691    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.592236    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.600576    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.600595    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.600611    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.600617    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600674    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600679    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600683    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:24.602499 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600686    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600690    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600693    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600696    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600699    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600702    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600705    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600708    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600711    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600715    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600718    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600722    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600725    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600727    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600730    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600733    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600736    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600739    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600742    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:24.602883 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600754    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600757    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600760    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600763    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600765    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600768    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600771    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600774    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600777    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600780    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600783    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600786    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600789    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600792    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600794    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600797    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600800    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600802    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600807    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600810    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:24.603391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600813    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600816    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600818    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600821    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600823    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600826    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600829    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600831    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600834    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600837    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600839    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600859    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600864    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600867    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600870    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600873    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600876    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600879    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600881    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:24.603889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600884    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600887    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600890    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600893    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600896    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600898    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600901    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600903    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600906    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600909    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600912    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600914    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600917    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600919    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600922    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600925    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600927    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600930    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600934    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600952    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:24.604391 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600955    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600958    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600962    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600965    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600968    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600971    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600974    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600977    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600980    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600982    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600985    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600987    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600991    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600993    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600996    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.600999    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601002    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:24.605002 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601004    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.601010    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601123    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601128    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601131    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601133    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601136    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601139    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601141    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601144    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601147    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601149    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601151    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601154    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601162    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:24.605481 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601164    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601167    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601169    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601172    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601175    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601178    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601180    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601183    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601186    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601188    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601191    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601194    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601196    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601200    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601202    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601205    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601208    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601210    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601213    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601216    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:24.605889 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601218    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601221    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601223    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601226    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601228    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601231    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601233    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601236    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601239    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601241    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601244    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601246    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601250    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601252    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601255    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601258    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601261    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601263    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601266    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601268    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:24.606472 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601271    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601273    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601276    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601279    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601281    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601284    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601287    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601289    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601292    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601294    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601297    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601300    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601303    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601305    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601308    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601310    2584 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601313    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601316    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601318    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601320    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:24.606978 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601323    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601327    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601330    2584 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601333    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601336    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601341    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601345    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601348    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601351    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601354    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601356    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601360    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601363    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601366    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601369    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601371    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601374    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601377    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601380    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:24.607485 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601383    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601385    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601388    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601390    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601393    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601395    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:24.601398    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.601403    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.601521    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.604073    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.605031    2584 server.go:1019] "Starting client certificate rotation"
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.605143    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.606048    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 20:58:24.608066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.607773    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 20:58:24.609002 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.608981    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 20:58:24.624602 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.624577    2584 log.go:25] "Validated CRI v1 runtime API"
Sep 10 20:58:24.627943 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.627926    2584 log.go:25] "Validated CRI v1 image API"
Sep 10 20:58:24.628783 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.628766    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 20:58:24.632280 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.632260    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7d0ee1ee-ffd7-4f37-8b46-6b356939a601:/dev/nvme0n1p3 affa99d5-be4d-4e57-b269-ed124410451b:/dev/nvme0n1p4]
Sep 10 20:58:24.632347 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.632279    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 20:58:24.639651 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.639528    2584 manager.go:216] Machine: {Timestamp:2026-09-10 20:58:24.638121428 +0000 UTC m=+0.369455981 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097068 MemoryCapacity:33164468224 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec278b24b2d41685886af05a55864f38 SystemUUID:ec278b24-b2d4-1685-886a-f05a55864f38 BootID:b3a6db9a-6efe-4e50-84ef-0026d9aaa8d4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:95:41:e7:2c:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:95:41:e7:2c:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:72:56:fd:6e:e4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164468224 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 20:58:24.639651 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.639638    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 20:58:24.639819 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.639730    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 20:58:24.639964 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.639946    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 20:58:24.643332 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.643306    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 20:58:24.643473 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.643335    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-196.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 20:58:24.644228 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.644218    2584 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 20:58:24.644261 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.644229    2584 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 20:58:24.644261 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.644237    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 20:58:24.644963 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.644953    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 20:58:24.645947 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.645936    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 20:58:24.650091 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650078    2584 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 20:58:24.650212 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650203    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 20:58:24.650241 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650232    2584 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 20:58:24.650269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650243    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 20:58:24.650269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650257    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 20:58:24.650269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650266    2584 kubelet.go:398] "Adding apiserver pod source"
Sep 10 20:58:24.650369 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650275    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 20:58:24.650749 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650727    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 20:58:24.650837 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.650752    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 20:58:24.653228 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.653209    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 20:58:24.656341 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.656326    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 20:58:24.656412 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.656356    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 20:58:24.658773 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658749    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 20:58:24.658773 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658771    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 20:58:24.658773 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658779    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658787    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658798    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658805    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658823    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658831    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658841    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658862    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658875    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 20:58:24.658972 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.658882    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 20:58:24.660440 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.660428    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 20:58:24.660500 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.660442    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 20:58:24.662356 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.662328    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 20:58:24.662467 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.662381    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 20:58:24.664357 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.664341    2584 server.go:1268] "Started kubelet"
Sep 10 20:58:24.664446 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.664426    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 20:58:24.664490 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.664419    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 20:58:24.664527 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.664491    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 20:58:24.665358 ip-10-0-128-196 systemd[1]: Started Kubernetes Kubelet.
Sep 10 20:58:24.666148 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.666134    2584 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 20:58:24.666286 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.666270    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 20:58:24.668770 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.668752    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 20:58:24.669292 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.669270    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-196.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 20:58:24.673366 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.673347    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 20:58:24.674026 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674010    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 20:58:24.674152 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.674134    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:24.674791 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674771    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 20:58:24.674791 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674791    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 20:58:24.674942 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674834    2584 factory.go:55] Registering systemd factory
Sep 10 20:58:24.674942 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674863    2584 factory.go:223] Registration of the systemd container factory successfully
Sep 10 20:58:24.674942 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674914    2584 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 20:58:24.674942 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674925    2584 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 20:58:24.674942 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.674931    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 20:58:24.675146 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.675089    2584 factory.go:153] Registering CRI-O factory
Sep 10 20:58:24.675146 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.675100    2584 factory.go:223] Registration of the crio container factory successfully
Sep 10 20:58:24.675243 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.675148    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 20:58:24.675243 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.675163    2584 factory.go:103] Registering Raw factory
Sep 10 20:58:24.675243 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.675173    2584 manager.go:1196] Started watching for new ooms in manager
Sep 10 20:58:24.675543 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.675532    2584 manager.go:349] Starting recovery of all containers
Sep 10 20:58:24.677143 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.677111    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 20:58:24.677397 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.677351    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 20:58:24.680304 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.677464    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc46ff9f4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.664312308 +0000 UTC m=+0.395646862,LastTimestamp:2026-09-10 20:58:24.664312308 +0000 UTC m=+0.395646862,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.687045 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.687026    2584 manager.go:354] Recovery completed
Sep 10 20:58:24.691497 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.691484    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:24.694115 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.694089    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:24.694163 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.694131    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:24.694163 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.694144    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:24.694775 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.694762    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 20:58:24.694775 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.694771    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:24.694894 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.694775    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 20:58:24.694894 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.694817    2584 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 20:58:24.695548 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.695535    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:24.695597 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.695561    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:24.695597 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.695574    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:24.696263 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.696096    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc636c159  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-196.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.694116697 +0000 UTC m=+0.425451253,LastTimestamp:2026-09-10 20:58:24.694116697 +0000 UTC m=+0.425451253,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.696361 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.696179    2584 policy_none.go:49] "None policy: Start"
Sep 10 20:58:24.696361 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.696344    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 20:58:24.696361 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.696355    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 20:58:24.697328 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.697317    2584 policy_none.go:47] "Start"
Sep 10 20:58:24.719736 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.719668    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc637108c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-196.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.694136972 +0000 UTC m=+0.425471525,LastTimestamp:2026-09-10 20:58:24.694136972 +0000 UTC m=+0.425471525,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.723493 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.723477    2584 manager.go:337] "Starting Device Plugin manager"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.723520    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.723531    2584 server.go:87] "Starting device plugin registration server"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.723872    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.723884    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.723983    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.724067    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.724075    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.724635    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.724684    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.732893    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc6374543  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-196.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.694150467 +0000 UTC m=+0.425485022,LastTimestamp:2026-09-10 20:58:24.694150467 +0000 UTC m=+0.425485022,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.738196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.738084    2584 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2620 into the system container \"/system.slice\": <nil>"
Sep 10 20:58:24.746160 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.746092    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-196.ec2.internal.18d4113cc636c159\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc636c159  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-196.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.694116697 +0000 UTC m=+0.425451253,LastTimestamp:2026-09-10 20:58:24.695548269 +0000 UTC m=+0.426882822,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.784730 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.784661    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-196.ec2.internal.18d4113cc637108c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc637108c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-196.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.694136972 +0000 UTC m=+0.425471525,LastTimestamp:2026-09-10 20:58:24.69556815 +0000 UTC m=+0.426902705,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.797352 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.797258    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-196.ec2.internal.18d4113cc6374543\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc6374543  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-196.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.694150467 +0000 UTC m=+0.425485022,LastTimestamp:2026-09-10 20:58:24.695578743 +0000 UTC m=+0.426913296,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.810887 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.810796    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-196.ec2.internal.18d4113cc81a5cd8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-196.ec2.internal,UID:ip-10-0-128-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-196.ec2.internal,},FirstTimestamp:2026-09-10 20:58:24.725810392 +0000 UTC m=+0.457144935,LastTimestamp:2026-09-10 20:58:24.725810392 +0000 UTC m=+0.457144935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-196.ec2.internal,}"
Sep 10 20:58:24.812720 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.812703    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-68wf9"
Sep 10 20:58:24.824774 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.824756    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:24.825062 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.825045    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-68wf9"
Sep 10 20:58:24.825729 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.825714    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:24.825804 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.825740    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:24.825804 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.825750    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:24.825804 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.825780    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-196.ec2.internal"
Sep 10 20:58:24.879345 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.879264    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-196.ec2.internal"
Sep 10 20:58:24.879345 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.879290    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-196.ec2.internal\": node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:24.887667 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.887643    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 20:58:24.888761 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.888740    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 20:58:24.888890 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.888802    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 20:58:24.888890 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.888830    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 20:58:24.889000 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.888949    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 20:58:24.891669 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.891652    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:24.963557 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:24.963535    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:24.989855 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.989818    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal"]
Sep 10 20:58:24.989957 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.989907    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:24.992322 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.992303    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:24.992438 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.992340    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:24.992438 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.992354    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:24.993727 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.993713    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:24.993920 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.993905    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:24.993979 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.993935    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:24.994488 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.994469    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:24.994583 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.994497    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:24.994583 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.994508    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:24.994583 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.994470    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:24.994583 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.994570    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:24.994583 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.994581    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:24.995677 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.995663    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:24.995730 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.995686    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:24.996327 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.996312    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:24.996400 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.996339    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:24.996400 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:24.996352    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:25.016362 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:25.016338    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-196.ec2.internal\" not found" node="ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.020626 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:25.020610    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-196.ec2.internal\" not found" node="ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.064466 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:25.064440    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:25.165509 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:25.165445    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:25.176871 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.176832    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b2a885bf71f7de756329dbbd36a414e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal\" (UID: \"b2a885bf71f7de756329dbbd36a414e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.176932 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.176877    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a885bf71f7de756329dbbd36a414e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal\" (UID: \"b2a885bf71f7de756329dbbd36a414e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.176932 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.176895    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89a6a8e54f676254b2e19e6c94fb7b2e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-196.ec2.internal\" (UID: \"89a6a8e54f676254b2e19e6c94fb7b2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.265536 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:25.265503    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:25.277923 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.277897    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b2a885bf71f7de756329dbbd36a414e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal\" (UID: \"b2a885bf71f7de756329dbbd36a414e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.277987 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.277928    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a885bf71f7de756329dbbd36a414e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal\" (UID: \"b2a885bf71f7de756329dbbd36a414e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.277987 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.277947    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89a6a8e54f676254b2e19e6c94fb7b2e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-196.ec2.internal\" (UID: \"89a6a8e54f676254b2e19e6c94fb7b2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.278084 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.277986    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/89a6a8e54f676254b2e19e6c94fb7b2e-config\") pod \"kube-apiserver-proxy-ip-10-0-128-196.ec2.internal\" (UID: \"89a6a8e54f676254b2e19e6c94fb7b2e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.278084 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.278019    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2a885bf71f7de756329dbbd36a414e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal\" (UID: \"b2a885bf71f7de756329dbbd36a414e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.278084 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.278025    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b2a885bf71f7de756329dbbd36a414e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal\" (UID: \"b2a885bf71f7de756329dbbd36a414e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.319058 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.319025    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.323711 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:25.323696    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:25.366341 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:25.366316    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:26.872470 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:26.872400    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:26.972895 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:26.972863    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:27.010380 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.010355    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 20:58:27.010831 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.010545    2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 20:58:27.069797 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.069771    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:27.073857 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:27.073838    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:27.173971 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:27.173941    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-196.ec2.internal\" not found"
Sep 10 20:58:27.208066 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.208044    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:27.254631 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.254597    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 20:53:26 +0000 UTC" deadline="2028-02-28 03:48:58.904150319 +0000 UTC"
Sep 10 20:58:27.254631 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.254627    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12846h50m31.649526626s"
Sep 10 20:58:27.279934 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.279912    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:27.284634 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:27.284603    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a6a8e54f676254b2e19e6c94fb7b2e.slice/crio-f31152dfef85ed34c3c23a0f841a50b7ffc5237b7b1e3894cb641ba8382b4687 WatchSource:0}: Error finding container f31152dfef85ed34c3c23a0f841a50b7ffc5237b7b1e3894cb641ba8382b4687: Status 404 returned error can't find the container with id f31152dfef85ed34c3c23a0f841a50b7ffc5237b7b1e3894cb641ba8382b4687
Sep 10 20:58:27.285248 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:27.285226    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a885bf71f7de756329dbbd36a414e3.slice/crio-72882cc1e35f1ab6dd52ebadabf7bb73370f99b41bc123be10b0b524a63a9593 WatchSource:0}: Error finding container 72882cc1e35f1ab6dd52ebadabf7bb73370f99b41bc123be10b0b524a63a9593: Status 404 returned error can't find the container with id 72882cc1e35f1ab6dd52ebadabf7bb73370f99b41bc123be10b0b524a63a9593
Sep 10 20:58:27.290354 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.290338    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 20:58:27.294836 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.294820    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 20:58:27.295675 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.295655    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal"
Sep 10 20:58:27.297984 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.297946    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal" event={"ID":"89a6a8e54f676254b2e19e6c94fb7b2e","Type":"ContainerStarted","Data":"f31152dfef85ed34c3c23a0f841a50b7ffc5237b7b1e3894cb641ba8382b4687"}
Sep 10 20:58:27.298959 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.298941    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal" event={"ID":"b2a885bf71f7de756329dbbd36a414e3","Type":"ContainerStarted","Data":"72882cc1e35f1ab6dd52ebadabf7bb73370f99b41bc123be10b0b524a63a9593"}
Sep 10 20:58:27.321222 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.321201    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 20:58:27.409707 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:27.409687    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:28.056364 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.056329    2584 apiserver.go:52] "Watching apiserver"
Sep 10 20:58:28.074879 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.074844    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 20:58:28.075351 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.075317    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-cs6qr","kube-system/global-pull-secret-syncer-dq9qc","kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx","openshift-image-registry/node-ca-sqjpl","openshift-multus/multus-dzwt9","openshift-multus/network-metrics-daemon-4k2zp","openshift-network-diagnostics/network-check-target-pw4qs","kube-system/konnectivity-agent-b4b5r","openshift-cluster-node-tuning-operator/tuned-sgqzl","openshift-dns/node-resolver-fz9wf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal","openshift-multus/multus-additional-cni-plugins-qlqrg","openshift-network-operator/iptables-alerter-g668d"]
Sep 10 20:58:28.077952 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.077888    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:28.078082 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.077987    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:28.079976 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.079953    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 20:58:28.080149 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.080126    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.082283 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.082259    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.084928 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.084674    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.086794    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.086814    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.086973    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.087185    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.087456    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.087483    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-225jt\""
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.087649    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.087840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.087802    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cbvg6\""
Sep 10 20:58:28.088236 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.088133    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 20:58:28.088750 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.088348    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 20:58:28.088750 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.088669    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.089780 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.089759    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.090699 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.090674    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.090823 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.090721    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.090823 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.090756    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/839bda7d-5447-459a-8141-501d7dd85d64-host-slash\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.090823 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.090786    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx6fx\" (UniqueName: \"kubernetes.io/projected/2f4353e4-9d2e-438d-968d-85d5039afa54-kube-api-access-jx6fx\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.090828    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-system-cni-dir\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.091369    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.091401    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2qsg\" (UniqueName: \"kubernetes.io/projected/30b5cecd-c292-4962-8043-843101bb70c5-kube-api-access-j2qsg\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.091442    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxp4z\" (UniqueName: \"kubernetes.io/projected/839bda7d-5447-459a-8141-501d7dd85d64-kube-api-access-dxp4z\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.091467    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-cnibin\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.091494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.091515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.091753 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.091553    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/839bda7d-5447-459a-8141-501d7dd85d64-iptables-alerter-script\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.092209 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.092023    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 20:58:28.093477 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.093457    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.095723 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.095703    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.098213 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.098197    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.099287 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.099250    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.099380 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.099331    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.099469 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.099452    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.099469 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.099464    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7znqc\""
Sep 10 20:58:28.099586 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.099464    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 20:58:28.099735 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.099714    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 20:58:28.100336 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.100317    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.100444 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.100394    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:28.101301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.101280    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 20:58:28.103153 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.103133    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.106398 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.106115    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.108145 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.108122    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:28.108246 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.108212    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:28.110237 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.110111    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v479g\""
Sep 10 20:58:28.110237 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.110130    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hj6np\""
Sep 10 20:58:28.110237 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.110177    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 20:58:28.110237 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.110189    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.110237 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.110197    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6hxkt\""
Sep 10 20:58:28.110521 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.110268    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 20:58:28.111206 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.111188    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 20:58:28.112863 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.112841    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 20:58:28.112964 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.112889    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.113094 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113072    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jp6cz\""
Sep 10 20:58:28.113507 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113293    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.113507 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113302    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.113507 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113296    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 20:58:28.113507 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113404    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.113744 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113592    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sxn8g\""
Sep 10 20:58:28.113744 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113602    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jwr4b\""
Sep 10 20:58:28.113744 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.113639    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.117655 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.117638    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 20:58:28.130493 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.130474    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4psvh"
Sep 10 20:58:28.142899 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.142878    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4psvh"
Sep 10 20:58:28.181625 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.181595    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192044    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysctl-conf\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192082    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-socket-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192107    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/35386900-a40f-41d0-9c53-fe52d51e775a-kubelet-config\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192133    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q95p7\" (UniqueName: \"kubernetes.io/projected/09521e42-9209-4a90-bead-2f4f4b6fdc18-kube-api-access-q95p7\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192155    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9f7f5cd3-168b-4020-a3be-79c741f85e9b-serviceca\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192178    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-node-log\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192199    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-var-lib-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.192269 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192228    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192276    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/35386900-a40f-41d0-9c53-fe52d51e775a-dbus\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192324    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f7f5cd3-168b-4020-a3be-79c741f85e9b-host\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192374    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlntj\" (UniqueName: \"kubernetes.io/projected/9f7f5cd3-168b-4020-a3be-79c741f85e9b-kube-api-access-nlntj\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192433    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-system-cni-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192468    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-etc-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192499    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192536    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09521e42-9209-4a90-bead-2f4f4b6fdc18-tmp-dir\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192555    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-cni-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192596    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2910372f-b1e0-4068-afda-01979c506ae9-multus-daemon-config\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192626    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2910372f-b1e0-4068-afda-01979c506ae9-cni-binary-copy\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192647    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovnkube-config\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192719    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-lib-modules\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.192772 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192770    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192796    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-cni-bin\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192814    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192828    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-slash\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192851    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-log-socket\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192890    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovn-node-metrics-cert\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192934    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-host\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192971    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/839bda7d-5447-459a-8141-501d7dd85d64-iptables-alerter-script\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.192997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-etc-selinux\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193020    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-cnibin\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193042    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-hostroot\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193065    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-multus-certs\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193091    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193097    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/09521e42-9209-4a90-bead-2f4f4b6fdc18-hosts-file\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193167    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-k8s-cni-cncf-io\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193194    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-etc-kubernetes\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193216    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-cni-bin\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.193414 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193244    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxp4z\" (UniqueName: \"kubernetes.io/projected/839bda7d-5447-459a-8141-501d7dd85d64-kube-api-access-dxp4z\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193273    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-netns\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193295    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdsh4\" (UniqueName: \"kubernetes.io/projected/66b3af61-0155-48c7-87a7-5cbde4107772-kube-api-access-pdsh4\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193409    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-systemd-units\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193453    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-cnibin\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193476    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193498    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-socket-dir-parent\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193540    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-kubelet\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193530    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-cnibin\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193562    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx4zn\" (UniqueName: \"kubernetes.io/projected/2910372f-b1e0-4068-afda-01979c506ae9-kube-api-access-rx4zn\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193603    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-run\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193640    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/839bda7d-5447-459a-8141-501d7dd85d64-iptables-alerter-script\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193653    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-registration-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.193708    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-systemd\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.194208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.194121    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-sys\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.194966 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.194940    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:28.195106 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.195091    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:28.695016772 +0000 UTC m=+3.020777674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:28.195675 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195646    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/839bda7d-5447-459a-8141-501d7dd85d64-host-slash\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.195771 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195703    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-device-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.195771 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195740    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-sys-fs\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.195862 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195784    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40-konnectivity-ca\") pod \"konnectivity-agent-b4b5r\" (UID: \"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40\") " pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.195862 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195826    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.195952 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195881    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-env-overrides\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.195952 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx6fx\" (UniqueName: \"kubernetes.io/projected/2f4353e4-9d2e-438d-968d-85d5039afa54-kube-api-access-jx6fx\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:28.196042 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.195967    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-system-cni-dir\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.196042 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196008    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/839bda7d-5447-459a-8141-501d7dd85d64-host-slash\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.196042 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2qsg\" (UniqueName: \"kubernetes.io/projected/30b5cecd-c292-4962-8043-843101bb70c5-kube-api-access-j2qsg\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.196170 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196086    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.196170 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.196170 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196159    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-modprobe-d\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.196305 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196187    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-var-lib-kubelet\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.196305 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196286    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chqnm\" (UniqueName: \"kubernetes.io/projected/e404e014-5f65-4b1a-aa8f-57945d3c08de-kube-api-access-chqnm\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.196395 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196317    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-cni-multus\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.196395 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196360    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-ovn\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.196395 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196382    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-cni-netd\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.196547 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b6l8\" (UniqueName: \"kubernetes.io/projected/e038fb84-0426-48e5-bf8d-38200bd7d6c8-kube-api-access-9b6l8\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.196547 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196535    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysconfig\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.196643 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196563    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66b3af61-0155-48c7-87a7-5cbde4107772-etc-tuned\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.196643 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196591    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-run-netns\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.196643 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196614    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-systemd\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.196774 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovnkube-script-lib\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.196896 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-kubernetes\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.196950 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.196999 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196967    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40-agent-certs\") pod \"konnectivity-agent-b4b5r\" (UID: \"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40\") " pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.196999 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.196992    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-conf-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.197094 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.197005    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30b5cecd-c292-4962-8043-843101bb70c5-system-cni-dir\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.197094 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.197024    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.197874 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.197438    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66b3af61-0155-48c7-87a7-5cbde4107772-tmp\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.197874 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.197487    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.197874 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.197524    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-kubelet\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.197874 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.197558    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysctl-d\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.197874 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.197641    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.198153 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.198101    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30b5cecd-c292-4962-8043-843101bb70c5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.209956 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.209934    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 20:58:28.213724 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.213702    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx6fx\" (UniqueName: \"kubernetes.io/projected/2f4353e4-9d2e-438d-968d-85d5039afa54-kube-api-access-jx6fx\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:28.213830 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.213705    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2qsg\" (UniqueName: \"kubernetes.io/projected/30b5cecd-c292-4962-8043-843101bb70c5-kube-api-access-j2qsg\") pod \"multus-additional-cni-plugins-qlqrg\" (UID: \"30b5cecd-c292-4962-8043-843101bb70c5\") " pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.220547 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.220525    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxp4z\" (UniqueName: \"kubernetes.io/projected/839bda7d-5447-459a-8141-501d7dd85d64-kube-api-access-dxp4z\") pod \"iptables-alerter-g668d\" (UID: \"839bda7d-5447-459a-8141-501d7dd85d64\") " pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.298669 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298638    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-kubelet\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.298843 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298676    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysctl-d\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.298843 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298692    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysctl-conf\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.298843 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298758    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-kubelet\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.298843 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298812    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-socket-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298848    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysctl-conf\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298858    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/35386900-a40f-41d0-9c53-fe52d51e775a-kubelet-config\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298885    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95p7\" (UniqueName: \"kubernetes.io/projected/09521e42-9209-4a90-bead-2f4f4b6fdc18-kube-api-access-q95p7\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298909    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/35386900-a40f-41d0-9c53-fe52d51e775a-kubelet-config\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298911    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9f7f5cd3-168b-4020-a3be-79c741f85e9b-serviceca\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298856    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysctl-d\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298948    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-node-log\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298993    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-var-lib-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298980    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-node-log\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/35386900-a40f-41d0-9c53-fe52d51e775a-dbus\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.299050 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299051    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-var-lib-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299057    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f7f5cd3-168b-4020-a3be-79c741f85e9b-host\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.298997    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-socket-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299082    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlntj\" (UniqueName: \"kubernetes.io/projected/9f7f5cd3-168b-4020-a3be-79c741f85e9b-kube-api-access-nlntj\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299106    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-system-cni-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299116    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/35386900-a40f-41d0-9c53-fe52d51e775a-dbus\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299129    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-etc-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299134    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f7f5cd3-168b-4020-a3be-79c741f85e9b-host\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09521e42-9209-4a90-bead-2f4f4b6fdc18-tmp-dir\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299181    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-cni-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2910372f-b1e0-4068-afda-01979c506ae9-multus-daemon-config\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299227    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2910372f-b1e0-4068-afda-01979c506ae9-cni-binary-copy\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299249    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-system-cni-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299254    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovnkube-config\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299286    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-lib-modules\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299292    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-cni-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9f7f5cd3-168b-4020-a3be-79c741f85e9b-serviceca\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299324    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.299564 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299333    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-etc-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299353    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-cni-bin\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-slash\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299398    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-log-socket\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovn-node-metrics-cert\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299461    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-host\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299486    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-etc-selinux\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299508    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-cnibin\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299550    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-hostroot\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299572    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-multus-certs\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299595    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/09521e42-9209-4a90-bead-2f4f4b6fdc18-hosts-file\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299628    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-k8s-cni-cncf-io\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299633    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09521e42-9209-4a90-bead-2f4f4b6fdc18-tmp-dir\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299678    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-etc-kubernetes\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299703    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-cni-bin\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299707    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-host\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299730    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-netns\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299754    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdsh4\" (UniqueName: \"kubernetes.io/projected/66b3af61-0155-48c7-87a7-5cbde4107772-kube-api-access-pdsh4\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.300475 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299781    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-systemd-units\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.299795    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299806    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-socket-dir-parent\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.299868    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret podName:35386900-a40f-41d0-9c53-fe52d51e775a nodeName:}" failed. No retries permitted until 2026-09-10 20:58:28.799849141 +0000 UTC m=+3.125610047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret") pod "global-pull-secret-syncer-dq9qc" (UID: "35386900-a40f-41d0-9c53-fe52d51e775a") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299884    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-kubelet\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299891    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovnkube-config\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299908    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx4zn\" (UniqueName: \"kubernetes.io/projected/2910372f-b1e0-4068-afda-01979c506ae9-kube-api-access-rx4zn\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299944    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2910372f-b1e0-4068-afda-01979c506ae9-multus-daemon-config\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-log-socket\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299957    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2910372f-b1e0-4068-afda-01979c506ae9-cni-binary-copy\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299986    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-run\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300015    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-etc-kubernetes\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300017    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-systemd-units\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-registration-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300055    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-etc-selinux\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300056    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-systemd\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.301276 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300084    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-sys\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300090    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-cnibin\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300100    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-netns\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-device-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300129    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-hostroot\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300057    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-cni-bin\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300156    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-multus-certs\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300193    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/09521e42-9209-4a90-bead-2f4f4b6fdc18-hosts-file\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300199    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-sys\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-device-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300240    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-systemd\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300265    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-run\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300285    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-cni-bin\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300284    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-kubelet\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300318    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-socket-dir-parent\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300321    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-run-k8s-cni-cncf-io\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.299437    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-lib-modules\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300341    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-sys-fs\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.302139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300357    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-registration-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300374    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40-konnectivity-ca\") pod \"konnectivity-agent-b4b5r\" (UID: \"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40\") " pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300403    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-sys-fs\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300444    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-env-overrides\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300467    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-slash\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300501    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300520    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-openvswitch\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300530    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300573    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-run-ovn-kubernetes\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300579    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e404e014-5f65-4b1a-aa8f-57945d3c08de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300586    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-modprobe-d\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300609    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-var-lib-kubelet\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300625    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chqnm\" (UniqueName: \"kubernetes.io/projected/e404e014-5f65-4b1a-aa8f-57945d3c08de-kube-api-access-chqnm\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300663    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-cni-multus\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300685    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-ovn\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300701    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-var-lib-kubelet\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300706    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-cni-netd\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300735    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-modprobe-d\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300752    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b6l8\" (UniqueName: \"kubernetes.io/projected/e038fb84-0426-48e5-bf8d-38200bd7d6c8-kube-api-access-9b6l8\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300766    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-ovn\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300779    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysconfig\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300805    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66b3af61-0155-48c7-87a7-5cbde4107772-etc-tuned\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300824    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-cni-netd\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300829    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-run-netns\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300854    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-systemd\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300858    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-host-var-lib-cni-multus\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300867    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-env-overrides\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300879    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovnkube-script-lib\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300904    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-kubernetes\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300927    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40-konnectivity-ca\") pod \"konnectivity-agent-b4b5r\" (UID: \"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40\") " pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300931    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-run-systemd\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300929    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40-agent-certs\") pod \"konnectivity-agent-b4b5r\" (UID: \"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40\") " pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-run-netns\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300981    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-kubernetes\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.303784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300982    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-conf-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.300936    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66b3af61-0155-48c7-87a7-5cbde4107772-etc-sysconfig\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.301038    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2910372f-b1e0-4068-afda-01979c506ae9-multus-conf-dir\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.301079    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.301108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66b3af61-0155-48c7-87a7-5cbde4107772-tmp\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.301124    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e038fb84-0426-48e5-bf8d-38200bd7d6c8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.301392    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovnkube-script-lib\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.303705    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40-agent-certs\") pod \"konnectivity-agent-b4b5r\" (UID: \"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40\") " pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.303758    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66b3af61-0155-48c7-87a7-5cbde4107772-etc-tuned\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.303803    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66b3af61-0155-48c7-87a7-5cbde4107772-tmp\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.304472 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.304255    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e038fb84-0426-48e5-bf8d-38200bd7d6c8-ovn-node-metrics-cert\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.314812 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.314747    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q95p7\" (UniqueName: \"kubernetes.io/projected/09521e42-9209-4a90-bead-2f4f4b6fdc18-kube-api-access-q95p7\") pod \"node-resolver-fz9wf\" (UID: \"09521e42-9209-4a90-bead-2f4f4b6fdc18\") " pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.318031 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.318004    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chqnm\" (UniqueName: \"kubernetes.io/projected/e404e014-5f65-4b1a-aa8f-57945d3c08de-kube-api-access-chqnm\") pod \"aws-ebs-csi-driver-node-vmrxx\" (UID: \"e404e014-5f65-4b1a-aa8f-57945d3c08de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.318929 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.318897    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx4zn\" (UniqueName: \"kubernetes.io/projected/2910372f-b1e0-4068-afda-01979c506ae9-kube-api-access-rx4zn\") pod \"multus-dzwt9\" (UID: \"2910372f-b1e0-4068-afda-01979c506ae9\") " pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.320586 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.320560    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdsh4\" (UniqueName: \"kubernetes.io/projected/66b3af61-0155-48c7-87a7-5cbde4107772-kube-api-access-pdsh4\") pod \"tuned-sgqzl\" (UID: \"66b3af61-0155-48c7-87a7-5cbde4107772\") " pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.320688 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.320630    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:28.321260 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.321235    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b6l8\" (UniqueName: \"kubernetes.io/projected/e038fb84-0426-48e5-bf8d-38200bd7d6c8-kube-api-access-9b6l8\") pod \"ovnkube-node-cs6qr\" (UID: \"e038fb84-0426-48e5-bf8d-38200bd7d6c8\") " pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.322673 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.322657    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlntj\" (UniqueName: \"kubernetes.io/projected/9f7f5cd3-168b-4020-a3be-79c741f85e9b-kube-api-access-nlntj\") pod \"node-ca-sqjpl\" (UID: \"9f7f5cd3-168b-4020-a3be-79c741f85e9b\") " pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.328072 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.328053    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:28.328072 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.328075    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:28.328222 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.328085    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hs8bb for pod openshift-network-diagnostics/network-check-target-pw4qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:28.328222 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.328137    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb podName:7f0707f7-cc69-4ad2-95f2-95e4a8068ac8 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:28.828121316 +0000 UTC m=+3.153882228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hs8bb" (UniqueName: "kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb") pod "network-check-target-pw4qs" (UID: "7f0707f7-cc69-4ad2-95f2-95e4a8068ac8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:28.391840 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.391810    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlqrg"
Sep 10 20:58:28.402460 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.402440    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g668d"
Sep 10 20:58:28.412155 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.412135    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx"
Sep 10 20:58:28.418632 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.418611    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sqjpl"
Sep 10 20:58:28.426797 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.426776    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dzwt9"
Sep 10 20:58:28.437448 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.437412    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:28.445140 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.445122    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:28.452030 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.452013    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sgqzl"
Sep 10 20:58:28.458731 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.458709    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fz9wf"
Sep 10 20:58:28.703280 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.703200    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:28.703453 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.703309    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:28.703453 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.703362    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:29.703344652 +0000 UTC m=+4.029105550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:28.803726 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.803695    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:28.803856 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.803831    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:28.803897 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.803887    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret podName:35386900-a40f-41d0-9c53-fe52d51e775a nodeName:}" failed. No retries permitted until 2026-09-10 20:58:29.803870727 +0000 UTC m=+4.129631649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret") pod "global-pull-secret-syncer-dq9qc" (UID: "35386900-a40f-41d0-9c53-fe52d51e775a") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:28.904734 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:28.904700    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:28.904901 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.904847    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:28.904901 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.904869    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:28.904901 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.904882    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hs8bb for pod openshift-network-diagnostics/network-check-target-pw4qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:28.905045 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:28.904948    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb podName:7f0707f7-cc69-4ad2-95f2-95e4a8068ac8 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:29.904928755 +0000 UTC m=+4.230689657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs8bb" (UniqueName: "kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb") pod "network-check-target-pw4qs" (UID: "7f0707f7-cc69-4ad2-95f2-95e4a8068ac8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:29.146686 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.146637    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 20:53:28 +0000 UTC" deadline="2028-05-29 10:25:41.177272032 +0000 UTC"
Sep 10 20:58:29.146686 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.146679    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15037h27m12.030595462s"
Sep 10 20:58:29.158629 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.158598    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2910372f_b1e0_4068_afda_01979c506ae9.slice/crio-bc0cbfe48d30d031bed0a71ef3a45c7aef8243b54ab4240c16bb094fec179e2e WatchSource:0}: Error finding container bc0cbfe48d30d031bed0a71ef3a45c7aef8243b54ab4240c16bb094fec179e2e: Status 404 returned error can't find the container with id bc0cbfe48d30d031bed0a71ef3a45c7aef8243b54ab4240c16bb094fec179e2e
Sep 10 20:58:29.160114 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.159986    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839bda7d_5447_459a_8141_501d7dd85d64.slice/crio-aa86ea34ac0e28962cd3113eb64192dbd624855d0a027d74f8a1730d08b59fac WatchSource:0}: Error finding container aa86ea34ac0e28962cd3113eb64192dbd624855d0a027d74f8a1730d08b59fac: Status 404 returned error can't find the container with id aa86ea34ac0e28962cd3113eb64192dbd624855d0a027d74f8a1730d08b59fac
Sep 10 20:58:29.163965 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.163940    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09521e42_9209_4a90_bead_2f4f4b6fdc18.slice/crio-b9eb7977e26d4a2a859f5b19eb9f12cd6ca270effab41febbae6a6575e1570ce WatchSource:0}: Error finding container b9eb7977e26d4a2a859f5b19eb9f12cd6ca270effab41febbae6a6575e1570ce: Status 404 returned error can't find the container with id b9eb7977e26d4a2a859f5b19eb9f12cd6ca270effab41febbae6a6575e1570ce
Sep 10 20:58:29.164814 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.164790    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7f5cd3_168b_4020_a3be_79c741f85e9b.slice/crio-8f8735cde507bdc494d5b02528dd07bc8c2a3acee15a79f23e8e0b13d7814973 WatchSource:0}: Error finding container 8f8735cde507bdc494d5b02528dd07bc8c2a3acee15a79f23e8e0b13d7814973: Status 404 returned error can't find the container with id 8f8735cde507bdc494d5b02528dd07bc8c2a3acee15a79f23e8e0b13d7814973
Sep 10 20:58:29.165702 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.165681    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b3af61_0155_48c7_87a7_5cbde4107772.slice/crio-679df44ccfe1d6a3abf587b9936c948466c1aa6d4094c4e23c2fc33690c03405 WatchSource:0}: Error finding container 679df44ccfe1d6a3abf587b9936c948466c1aa6d4094c4e23c2fc33690c03405: Status 404 returned error can't find the container with id 679df44ccfe1d6a3abf587b9936c948466c1aa6d4094c4e23c2fc33690c03405
Sep 10 20:58:29.166680 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.166522    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode038fb84_0426_48e5_bf8d_38200bd7d6c8.slice/crio-52c51598e7e8faebb73e62b3986dc8cd6a0f6ef39663eb2ea4f8493f6d663656 WatchSource:0}: Error finding container 52c51598e7e8faebb73e62b3986dc8cd6a0f6ef39663eb2ea4f8493f6d663656: Status 404 returned error can't find the container with id 52c51598e7e8faebb73e62b3986dc8cd6a0f6ef39663eb2ea4f8493f6d663656
Sep 10 20:58:29.167790 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.167768    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b5cecd_c292_4962_8043_843101bb70c5.slice/crio-e86220039f45963594625d75587b992c0fed6eddca8d04b0a8c9a329c0152a5a WatchSource:0}: Error finding container e86220039f45963594625d75587b992c0fed6eddca8d04b0a8c9a329c0152a5a: Status 404 returned error can't find the container with id e86220039f45963594625d75587b992c0fed6eddca8d04b0a8c9a329c0152a5a
Sep 10 20:58:29.169100 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.169077    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4c4bf2_a6f0_4bb4_bbf7_f55373d45b40.slice/crio-3b06b8140c531446024b543d6ff0ce12b74f4d9c7d4ac8d1962b4c8fc4b4896e WatchSource:0}: Error finding container 3b06b8140c531446024b543d6ff0ce12b74f4d9c7d4ac8d1962b4c8fc4b4896e: Status 404 returned error can't find the container with id 3b06b8140c531446024b543d6ff0ce12b74f4d9c7d4ac8d1962b4c8fc4b4896e
Sep 10 20:58:29.169791 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:58:29.169770    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode404e014_5f65_4b1a_aa8f_57945d3c08de.slice/crio-1e4990773021c6f71f1d435aff93813ea3876c3b6b5cedbc0d63d27090b23f91 WatchSource:0}: Error finding container 1e4990773021c6f71f1d435aff93813ea3876c3b6b5cedbc0d63d27090b23f91: Status 404 returned error can't find the container with id 1e4990773021c6f71f1d435aff93813ea3876c3b6b5cedbc0d63d27090b23f91
Sep 10 20:58:29.295539 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.295358    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:29.295674 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.295634    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:29.305077 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.305042    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"52c51598e7e8faebb73e62b3986dc8cd6a0f6ef39663eb2ea4f8493f6d663656"}
Sep 10 20:58:29.306170 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.306148    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sgqzl" event={"ID":"66b3af61-0155-48c7-87a7-5cbde4107772","Type":"ContainerStarted","Data":"679df44ccfe1d6a3abf587b9936c948466c1aa6d4094c4e23c2fc33690c03405"}
Sep 10 20:58:29.307335 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.307301    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sqjpl" event={"ID":"9f7f5cd3-168b-4020-a3be-79c741f85e9b","Type":"ContainerStarted","Data":"8f8735cde507bdc494d5b02528dd07bc8c2a3acee15a79f23e8e0b13d7814973"}
Sep 10 20:58:29.308430 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.308394    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g668d" event={"ID":"839bda7d-5447-459a-8141-501d7dd85d64","Type":"ContainerStarted","Data":"aa86ea34ac0e28962cd3113eb64192dbd624855d0a027d74f8a1730d08b59fac"}
Sep 10 20:58:29.309479 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.309442    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dzwt9" event={"ID":"2910372f-b1e0-4068-afda-01979c506ae9","Type":"ContainerStarted","Data":"bc0cbfe48d30d031bed0a71ef3a45c7aef8243b54ab4240c16bb094fec179e2e"}
Sep 10 20:58:29.311554 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.311532    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal" event={"ID":"89a6a8e54f676254b2e19e6c94fb7b2e","Type":"ContainerStarted","Data":"399639fb0bc13539dfc56a45456c943d64a3ec1e52bc7ac96ce576089d53f2e5"}
Sep 10 20:58:29.312728 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.312711    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx" event={"ID":"e404e014-5f65-4b1a-aa8f-57945d3c08de","Type":"ContainerStarted","Data":"1e4990773021c6f71f1d435aff93813ea3876c3b6b5cedbc0d63d27090b23f91"}
Sep 10 20:58:29.313854 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.313836    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fz9wf" event={"ID":"09521e42-9209-4a90-bead-2f4f4b6fdc18","Type":"ContainerStarted","Data":"b9eb7977e26d4a2a859f5b19eb9f12cd6ca270effab41febbae6a6575e1570ce"}
Sep 10 20:58:29.314886 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.314866    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b4b5r" event={"ID":"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40","Type":"ContainerStarted","Data":"3b06b8140c531446024b543d6ff0ce12b74f4d9c7d4ac8d1962b4c8fc4b4896e"}
Sep 10 20:58:29.315951 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.315933    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerStarted","Data":"e86220039f45963594625d75587b992c0fed6eddca8d04b0a8c9a329c0152a5a"}
Sep 10 20:58:29.336802 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.336763    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-196.ec2.internal" podStartSLOduration=2.336748103 podStartE2EDuration="2.336748103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:29.334153603 +0000 UTC m=+3.659914524" watchObservedRunningTime="2026-09-10 20:58:29.336748103 +0000 UTC m=+3.662509023"
Sep 10 20:58:29.709619 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.709527    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:29.709784 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.709698    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:29.709784 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.709767    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:31.709747358 +0000 UTC m=+6.035508295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:29.810620 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.810581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:29.810804 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.810723    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:29.810804 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.810782    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret podName:35386900-a40f-41d0-9c53-fe52d51e775a nodeName:}" failed. No retries permitted until 2026-09-10 20:58:31.810764839 +0000 UTC m=+6.136525742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret") pod "global-pull-secret-syncer-dq9qc" (UID: "35386900-a40f-41d0-9c53-fe52d51e775a") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:29.911582 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:29.911543    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:29.911777 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.911758    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:29.911845 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.911788    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:29.911845 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.911802    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hs8bb for pod openshift-network-diagnostics/network-check-target-pw4qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:29.911950 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:29.911861    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb podName:7f0707f7-cc69-4ad2-95f2-95e4a8068ac8 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:31.911843682 +0000 UTC m=+6.237604587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs8bb" (UniqueName: "kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb") pod "network-check-target-pw4qs" (UID: "7f0707f7-cc69-4ad2-95f2-95e4a8068ac8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:30.147521 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:30.147389    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 20:53:28 +0000 UTC" deadline="2028-06-27 06:32:59.736600906 +0000 UTC"
Sep 10 20:58:30.147521 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:30.147463    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15729h34m29.589144575s"
Sep 10 20:58:30.297766 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:30.297226    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:30.297766 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:30.297351    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:30.298209 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:30.298064    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:30.298209 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:30.298175    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:30.369819 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:30.369783    2584 generic.go:356] "Generic (PLEG): container finished" podID="b2a885bf71f7de756329dbbd36a414e3" containerID="3e0167f08856bc58577f114ab9af0cee673624682aaf784f2d895b947cd1b678" exitCode=0
Sep 10 20:58:30.370851 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:30.370814    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal" event={"ID":"b2a885bf71f7de756329dbbd36a414e3","Type":"ContainerDied","Data":"3e0167f08856bc58577f114ab9af0cee673624682aaf784f2d895b947cd1b678"}
Sep 10 20:58:31.294888 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:31.294799    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:31.295460 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.294945    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:31.397379 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:31.397341    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal" event={"ID":"b2a885bf71f7de756329dbbd36a414e3","Type":"ContainerStarted","Data":"b9779fa87f9400e47d2f726a7dbb6068ea58a15cbe84122813f8b902bd9d7cf4"}
Sep 10 20:58:31.728384 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:31.728300    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:31.728635 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.728522    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:31.728635 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.728601    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:35.728580181 +0000 UTC m=+10.054341097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:31.828878 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:31.828832    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:31.829065 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.829048    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:31.829126 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.829113    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret podName:35386900-a40f-41d0-9c53-fe52d51e775a nodeName:}" failed. No retries permitted until 2026-09-10 20:58:35.829095287 +0000 UTC m=+10.154856203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret") pod "global-pull-secret-syncer-dq9qc" (UID: "35386900-a40f-41d0-9c53-fe52d51e775a") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:31.930002 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:31.929964    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:31.930218 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.930143    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:31.930218 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.930163    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:31.930218 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.930179    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hs8bb for pod openshift-network-diagnostics/network-check-target-pw4qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:31.930430 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:31.930242    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb podName:7f0707f7-cc69-4ad2-95f2-95e4a8068ac8 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:35.930224418 +0000 UTC m=+10.255985343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs8bb" (UniqueName: "kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb") pod "network-check-target-pw4qs" (UID: "7f0707f7-cc69-4ad2-95f2-95e4a8068ac8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:32.295302 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:32.295268    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:32.295949 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:32.295399    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:32.295949 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:32.295475    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:32.295949 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:32.295557    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:33.295341 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:33.294790    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:33.295341 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:33.294951    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:34.295367 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:34.295270    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:34.295821 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:34.295431    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:34.295821 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:34.295528    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:34.295821 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:34.295658    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:35.295004 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:35.294962    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:35.295174 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.295116    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:35.763197 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:35.763108    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:35.763687 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.763269    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:35.763687 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.763372    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:43.763323266 +0000 UTC m=+18.089084169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:35.863985 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:35.863949    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:35.864206 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.864141    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:35.864283 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.864217    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret podName:35386900-a40f-41d0-9c53-fe52d51e775a nodeName:}" failed. No retries permitted until 2026-09-10 20:58:43.864197699 +0000 UTC m=+18.189958616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret") pod "global-pull-secret-syncer-dq9qc" (UID: "35386900-a40f-41d0-9c53-fe52d51e775a") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:35.964924 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:35.964876    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:35.965109 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.965082    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:35.965109 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.965109    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:35.965293 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.965124    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hs8bb for pod openshift-network-diagnostics/network-check-target-pw4qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:35.965293 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:35.965187    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb podName:7f0707f7-cc69-4ad2-95f2-95e4a8068ac8 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:43.965167768 +0000 UTC m=+18.290928689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs8bb" (UniqueName: "kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb") pod "network-check-target-pw4qs" (UID: "7f0707f7-cc69-4ad2-95f2-95e4a8068ac8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:36.295698 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:36.295658    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:36.295849 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:36.295744    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:36.295849 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:36.295790    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:36.295849 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:36.295808    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:37.295368 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:37.295332    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:37.295876 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:37.295499    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:38.295524 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:38.295493    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:38.295992 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:38.295628    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:38.296067 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:38.296006    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:38.296117 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:38.296095    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:39.294733 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:39.294696    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:39.294916 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:39.294826    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:40.294985 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:40.294936    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:40.294985 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:40.294974    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:40.295524 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:40.295080    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:40.295524 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:40.295204    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:41.294848 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:41.294815    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:41.295042 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:41.294948    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:42.294907 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:42.294866    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:42.295056 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:42.295007    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:42.295335 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:42.295052    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:42.295335 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:42.295133    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:43.295001 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:43.294965    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:43.295206 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:43.295112    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:43.818199 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:43.818160    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:43.818364 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:43.818296    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:43.818364 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:43.818360    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:59.818338998 +0000 UTC m=+34.144099898 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:43.919196 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:43.919158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:43.919370 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:43.919321    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:43.919454 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:43.919398    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret podName:35386900-a40f-41d0-9c53-fe52d51e775a nodeName:}" failed. No retries permitted until 2026-09-10 20:58:59.919377733 +0000 UTC m=+34.245138644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret") pod "global-pull-secret-syncer-dq9qc" (UID: "35386900-a40f-41d0-9c53-fe52d51e775a") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:44.019885 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:44.019837    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:44.020025 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:44.020001    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:44.020025 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:44.020023    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:44.020117 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:44.020033    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hs8bb for pod openshift-network-diagnostics/network-check-target-pw4qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:44.020117 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:44.020092    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb podName:7f0707f7-cc69-4ad2-95f2-95e4a8068ac8 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:00.020076289 +0000 UTC m=+34.345837190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs8bb" (UniqueName: "kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb") pod "network-check-target-pw4qs" (UID: "7f0707f7-cc69-4ad2-95f2-95e4a8068ac8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:44.294868 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:44.294788    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:44.295015 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:44.294918    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:44.295015 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:44.295001    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:44.295130 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:44.295108    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:45.294632 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:45.294599    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:45.295079 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:45.294727    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:46.295514 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:46.295471    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:46.295969 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:46.295592    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:46.295969 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:46.295634    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:46.295969 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:46.295724    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:47.295810 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.295404    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:47.296367 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:47.295909    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:47.434151 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.434061    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b4b5r" event={"ID":"7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40","Type":"ContainerStarted","Data":"ee4eeeabfed49f459a3cf932c8d59b90ef60964f45f422fa55f08a2d0c111ac2"}
Sep 10 20:58:47.436092 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.436064    2584 generic.go:356] "Generic (PLEG): container finished" podID="30b5cecd-c292-4962-8043-843101bb70c5" containerID="e9593afb567daade80fdfcb5b5c5604d893f9bd16210189e8af306732787d22f" exitCode=0
Sep 10 20:58:47.436207 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.436156    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerDied","Data":"e9593afb567daade80fdfcb5b5c5604d893f9bd16210189e8af306732787d22f"}
Sep 10 20:58:47.439533 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.439514    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 20:58:47.439934 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.439910    2584 generic.go:356] "Generic (PLEG): container finished" podID="e038fb84-0426-48e5-bf8d-38200bd7d6c8" containerID="9fe80228a504472fa8d4b954da4f2325e56cfbbc6a51ca2e1d0b4ab2daf03cf3" exitCode=1
Sep 10 20:58:47.440037 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.439988    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"e388114c95a98376e5637227fb20fbd00cfa1bb30de7fc590e7df3a5b71f66c3"}
Sep 10 20:58:47.440095 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.440063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"01f42258c72577c4c765637bf5a5194ce88d898b5526b13e5cba595a001038c8"}
Sep 10 20:58:47.440134 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.440099    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"ec941bc129614dc5e5df349b2b0fb5855a45fe0b077e61e87c9ab716573dd0ea"}
Sep 10 20:58:47.440188 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.440137    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"a2544f176e65e384624221aa7be478206af73fde646d639f2622cab966750350"}
Sep 10 20:58:47.440188 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.440180    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerDied","Data":"9fe80228a504472fa8d4b954da4f2325e56cfbbc6a51ca2e1d0b4ab2daf03cf3"}
Sep 10 20:58:47.440285 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.440229    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"199690c6689daed8eb2a0ad012cb36914fceb5d2d7e90d56e49ba0152b09823e"}
Sep 10 20:58:47.441627 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.441607    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sgqzl" event={"ID":"66b3af61-0155-48c7-87a7-5cbde4107772","Type":"ContainerStarted","Data":"ef22b9ba525669465c2a1cdeef08e01ff741b8e98bd32c5840c9d3d655cabe30"}
Sep 10 20:58:47.443247 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.443228    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sqjpl" event={"ID":"9f7f5cd3-168b-4020-a3be-79c741f85e9b","Type":"ContainerStarted","Data":"8e341d91ce1457b18c262226289834e3e3780d22a434603c959b06a2c442b201"}
Sep 10 20:58:47.444692 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.444671    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dzwt9" event={"ID":"2910372f-b1e0-4068-afda-01979c506ae9","Type":"ContainerStarted","Data":"4b588f9b82196b296286d5ab2469cf820a4c3159bab3296c82e5752fe2a1216d"}
Sep 10 20:58:47.446038 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.446023    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx" event={"ID":"e404e014-5f65-4b1a-aa8f-57945d3c08de","Type":"ContainerStarted","Data":"6f8ae679be80a4998141259f73495cedf98a897f0f0b3610d7b3afe225adf036"}
Sep 10 20:58:47.447293 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.447277    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fz9wf" event={"ID":"09521e42-9209-4a90-bead-2f4f4b6fdc18","Type":"ContainerStarted","Data":"1b1a73728722d61b66f3bb93813d30502ef7ce852679b1031781ad4171f66607"}
Sep 10 20:58:47.456875 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.456828    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-196.ec2.internal" podStartSLOduration=20.456813201 podStartE2EDuration="20.456813201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:31.42615582 +0000 UTC m=+5.751916742" watchObservedRunningTime="2026-09-10 20:58:47.456813201 +0000 UTC m=+21.782574124"
Sep 10 20:58:47.482712 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.482670    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b4b5r" podStartSLOduration=7.498525797 podStartE2EDuration="21.482654088s" totalImagesPullingTime="13.984128291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:47.456772449 +0000 UTC m=+21.782533371" watchObservedRunningTime="2026-09-10 20:58:47.482654088 +0000 UTC m=+21.808415061"
Sep 10 20:58:47.523728 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.523678    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fz9wf" podStartSLOduration=7.533378159 podStartE2EDuration="21.52366274s" totalImagesPullingTime="13.990284581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:47.483277284 +0000 UTC m=+21.809038204" watchObservedRunningTime="2026-09-10 20:58:47.52366274 +0000 UTC m=+21.849423661"
Sep 10 20:58:47.552203 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.552160    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sqjpl" podStartSLOduration=4.147057512 podStartE2EDuration="21.552145859s" totalImagesPullingTime="17.405088347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:47.547331499 +0000 UTC m=+21.873092433" watchObservedRunningTime="2026-09-10 20:58:47.552145859 +0000 UTC m=+21.877906779"
Sep 10 20:58:47.579811 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.579761    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dzwt9" podStartSLOduration=3.921684409 podStartE2EDuration="21.579739759s" totalImagesPullingTime="17.65805535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:47.579543778 +0000 UTC m=+21.905304722" watchObservedRunningTime="2026-09-10 20:58:47.579739759 +0000 UTC m=+21.905500685"
Sep 10 20:58:47.719211 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:47.719187    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 20:58:48.141552 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.141383    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T20:58:47.719203979Z","UUID":"c5e75895-06fb-40df-a86d-ed5d228fbedd","Handler":null,"Name":"","Endpoint":""}
Sep 10 20:58:48.143198 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.143171    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 20:58:48.143334 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.143215    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 20:58:48.295212 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.295179    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:48.295371 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:48.295305    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:48.295371 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.295361    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:48.295516 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:48.295484    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:48.454146 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.453924    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g668d" event={"ID":"839bda7d-5447-459a-8141-501d7dd85d64","Type":"ContainerStarted","Data":"e5e814b60d67594e41db33fe8ed925fbe14067670e6568a97f61c72a0fbd6cac"}
Sep 10 20:58:48.456736 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.456698    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx" event={"ID":"e404e014-5f65-4b1a-aa8f-57945d3c08de","Type":"ContainerStarted","Data":"b6fcfcd00ea35c658184d8a95ccd3a0408f193fe7edff706bf96567c4d2a1de8"}
Sep 10 20:58:48.482932 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.482875    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sgqzl" podStartSLOduration=5.079246321 podStartE2EDuration="22.482853229s" totalImagesPullingTime="17.403606908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:47.642246427 +0000 UTC m=+21.968007349" watchObservedRunningTime="2026-09-10 20:58:48.482853229 +0000 UTC m=+22.808614152"
Sep 10 20:58:48.483228 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:48.483192    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g668d" podStartSLOduration=5.123274999 podStartE2EDuration="22.483182027s" totalImagesPullingTime="17.359907028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:48.482968775 +0000 UTC m=+22.808729696" watchObservedRunningTime="2026-09-10 20:58:48.483182027 +0000 UTC m=+22.808942949"
Sep 10 20:58:49.295202 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:49.295170    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:49.295336 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:49.295294    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:49.463225 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:49.463198    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 20:58:49.463759 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:49.463719    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"ad61fda903bf5325aa4e803fecf3026334beebe4b17bbc31d912ac924bd5f669"}
Sep 10 20:58:49.466129 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:49.466104    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx" event={"ID":"e404e014-5f65-4b1a-aa8f-57945d3c08de","Type":"ContainerStarted","Data":"e0e8f260b4a4cd1b7b1a1f794aac69da6c1362c77b79e3b04d9ebd835cfce800"}
Sep 10 20:58:49.500599 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:49.500550    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vmrxx" podStartSLOduration=4.625189656 podStartE2EDuration="23.500530779s" totalImagesPullingTime="18.875341123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:49.50003084 +0000 UTC m=+23.825791774" watchObservedRunningTime="2026-09-10 20:58:49.500530779 +0000 UTC m=+23.826291903"
Sep 10 20:58:50.294784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:50.294588    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:50.295057 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:50.294611    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:50.295057 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:50.294864    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:50.295057 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:50.294921    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:50.587821 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:50.587732    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:50.588591 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:50.588565    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:51.295104 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:51.295074    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:51.295326 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:51.295205    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:52.295175 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.294890    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:52.295579 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.294924    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:52.295579 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:52.295227    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:52.295579 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:52.295284    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:52.475339 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.475277    2584 generic.go:356] "Generic (PLEG): container finished" podID="30b5cecd-c292-4962-8043-843101bb70c5" containerID="cf03bdb5d79858954c9ca7c7f334bd9e5d1dc4b0c29537bb86dd40f49df70568" exitCode=0
Sep 10 20:58:52.475508 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.475370    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerDied","Data":"cf03bdb5d79858954c9ca7c7f334bd9e5d1dc4b0c29537bb86dd40f49df70568"}
Sep 10 20:58:52.479208 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.479193    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 20:58:52.479568 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.479548    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"813d69e30fb0547641f5b8ee2b6d353583ccb9e4ba1760b0226c55a697934b7c"}
Sep 10 20:58:52.479805 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.479784    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:52.479992 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.479921    2584 scope.go:117] "RemoveContainer" containerID="9fe80228a504472fa8d4b954da4f2325e56cfbbc6a51ca2e1d0b4ab2daf03cf3"
Sep 10 20:58:52.496982 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:52.496953    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:53.295597 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.295555    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:53.296148 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:53.295676    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:53.342062 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.342027    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:53.342223 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.342180    2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 20:58:53.342600 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.342581    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b4b5r"
Sep 10 20:58:53.484443 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.484343    2584 generic.go:356] "Generic (PLEG): container finished" podID="30b5cecd-c292-4962-8043-843101bb70c5" containerID="9cadb9bb7d606b94e1aa81e6432ebd8a1fc14b5e3516f61f0cc36dcfb2a30fe3" exitCode=0
Sep 10 20:58:53.484628 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.484452    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerDied","Data":"9cadb9bb7d606b94e1aa81e6432ebd8a1fc14b5e3516f61f0cc36dcfb2a30fe3"}
Sep 10 20:58:53.488898 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.488879    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 20:58:53.489301 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.489277    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" event={"ID":"e038fb84-0426-48e5-bf8d-38200bd7d6c8","Type":"ContainerStarted","Data":"2513909bd5b515c98d99d07cfb257b305c2706a514326ac5611462f0197f3a9b"}
Sep 10 20:58:53.489413 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.489382    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:53.489579 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.489553    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:53.504534 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.504513    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:58:53.571782 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.571730    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr" podStartSLOduration=10.111533844 podStartE2EDuration="27.571716475s" totalImagesPullingTime="17.460182631s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:53.571692354 +0000 UTC m=+27.897453275" watchObservedRunningTime="2026-09-10 20:58:53.571716475 +0000 UTC m=+27.897477399"
Sep 10 20:58:53.792728 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.792641    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dq9qc"]
Sep 10 20:58:53.792879 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.792790    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:53.792952 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:53.792900    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:53.800267 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.800236    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4k2zp"]
Sep 10 20:58:53.800382 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.800342    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:53.800462 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:53.800441    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:53.812282 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.812243    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pw4qs"]
Sep 10 20:58:53.812453 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:53.812372    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:53.812514 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:53.812491    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:54.494604 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:54.494523    2584 generic.go:356] "Generic (PLEG): container finished" podID="30b5cecd-c292-4962-8043-843101bb70c5" containerID="606b4161a65620dc63e3590ec8faeae52065fc825fd6a2b3e9713babc83e3d12" exitCode=0
Sep 10 20:58:54.495081 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:54.494608    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerDied","Data":"606b4161a65620dc63e3590ec8faeae52065fc825fd6a2b3e9713babc83e3d12"}
Sep 10 20:58:55.294642 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:55.294604    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:55.294833 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:55.294604    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:55.294833 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:55.294718    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:55.294833 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:55.294812    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:55.294988 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:55.294604    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:55.294988 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:55.294929    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:57.294945 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:57.294914    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:57.295479 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:57.295036    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:57.295479 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:57.295053    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:57.295479 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:57.295043    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:57.295479 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:57.295141    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:57.295479 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:57.295240    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:59.294927 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.294887    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:59.294927 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.294924    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:58:59.295349 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.294903    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:59.295349 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.295029    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4k2zp" podUID="2f4353e4-9d2e-438d-968d-85d5039afa54"
Sep 10 20:58:59.295349 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.295127    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dq9qc" podUID="35386900-a40f-41d0-9c53-fe52d51e775a"
Sep 10 20:58:59.295349 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.295214    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pw4qs" podUID="7f0707f7-cc69-4ad2-95f2-95e4a8068ac8"
Sep 10 20:58:59.509345 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.509311    2584 generic.go:356] "Generic (PLEG): container finished" podID="30b5cecd-c292-4962-8043-843101bb70c5" containerID="5cdc8056639d3460407c11a23100e5e2398a8b83b01abf433c225a315377c63f" exitCode=0
Sep 10 20:58:59.509528 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.509379    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerDied","Data":"5cdc8056639d3460407c11a23100e5e2398a8b83b01abf433c225a315377c63f"}
Sep 10 20:58:59.523136 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.523112    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-196.ec2.internal" event="NodeReady"
Sep 10 20:58:59.523272 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.523249    2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 20:58:59.618480 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.618380    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dc64c7464-jzhls"]
Sep 10 20:58:59.635041 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.635018    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.645073 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.645044    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 20:58:59.646026 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.646004    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dc64c7464-jzhls"]
Sep 10 20:58:59.646131 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.646055    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 20:58:59.646131 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.646055    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 20:58:59.656626 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.656598    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bbgkn\""
Sep 10 20:58:59.657214 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.657193    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8x4fw"]
Sep 10 20:58:59.676030 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.676005    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.677591 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.677565    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q2w8b"]
Sep 10 20:58:59.678582 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.678562    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 20:58:59.678677 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.678602    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 20:58:59.678750 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.678733    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vl2hm\""
Sep 10 20:58:59.679235 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.679210    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 20:58:59.692351 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.692333    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:58:59.703053 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.703027    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 20:58:59.703265 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.703133    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 20:58:59.703265 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.703133    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 20:58:59.703265 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.703251    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tkv7t\""
Sep 10 20:58:59.704329 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.704308    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2w8b"]
Sep 10 20:58:59.714140 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.714118    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8x4fw"]
Sep 10 20:58:59.720731 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.720707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:58:59.735081 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg24n\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-kube-api-access-fg24n\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.735172 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735098    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c5445ae-52bb-4a45-a3e8-80b742b26b21-ca-trust-extracted\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.735172 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735124    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-trusted-ca\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.735245 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735171    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-certificates\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.735287 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735268    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-installation-pull-secrets\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.735321 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735297    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-image-registry-private-configuration\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.735366 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735326    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-bound-sa-token\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.735403 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.735365    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.835810 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835775    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-installation-pull-secrets\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.835810 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835819    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-image-registry-private-configuration\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835846    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-config-volume\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835873    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-bound-sa-token\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835903    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835930    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg24n\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-kube-api-access-fg24n\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835951    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.835976    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnmbs\" (UniqueName: \"kubernetes.io/projected/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-kube-api-access-rnmbs\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836024    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c5445ae-52bb-4a45-a3e8-80b742b26b21-ca-trust-extracted\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836067    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-trusted-ca\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836087 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.836033    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.836101    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dc64c7464-jzhls: secret "image-registry-tls" not found
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836119    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-certificates\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836146    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/947f3e28-930c-467e-8de6-356a58785659-cert\") pod \"ingress-canary-q2w8b\" (UID: \"947f3e28-930c-467e-8de6-356a58785659\") " pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.836175    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls podName:5c5445ae-52bb-4a45-a3e8-80b742b26b21 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:00.336153403 +0000 UTC m=+34.661914316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls") pod "image-registry-5dc64c7464-jzhls" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21") : secret "image-registry-tls" not found
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836358    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-tmp-dir\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836397    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c5445ae-52bb-4a45-a3e8-80b742b26b21-ca-trust-extracted\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836404    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:58:59.836496 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.836492    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:59.836784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7psg\" (UniqueName: \"kubernetes.io/projected/947f3e28-930c-467e-8de6-356a58785659-kube-api-access-g7psg\") pod \"ingress-canary-q2w8b\" (UID: \"947f3e28-930c-467e-8de6-356a58785659\") " pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:58:59.836784 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.836545    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:31.836528008 +0000 UTC m=+66.162288923 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:59.836784 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.836701    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-certificates\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.837062 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.837045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-trusted-ca\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.839992 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.839957    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-installation-pull-secrets\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.839992 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.839957    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-image-registry-private-configuration\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.853462 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.853439    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-bound-sa-token\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.853556 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.853532    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg24n\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-kube-api-access-fg24n\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:58:59.888408 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.888318    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:58:59.937299 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-tmp-dir\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.937437 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7psg\" (UniqueName: \"kubernetes.io/projected/947f3e28-930c-467e-8de6-356a58785659-kube-api-access-g7psg\") pod \"ingress-canary-q2w8b\" (UID: \"947f3e28-930c-467e-8de6-356a58785659\") " pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:58:59.937437 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937337    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-config-volume\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.937541 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.937576 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937559    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnmbs\" (UniqueName: \"kubernetes.io/projected/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-kube-api-access-rnmbs\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.937618 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937595    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:58:59.937673 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/947f3e28-930c-467e-8de6-356a58785659-cert\") pod \"ingress-canary-q2w8b\" (UID: \"947f3e28-930c-467e-8de6-356a58785659\") " pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:58:59.937725 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.937669    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:58:59.937725 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937676    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-tmp-dir\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.937816 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.937736    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls podName:a94dabbc-b8ff-4236-9bdd-cc3b21d92c38 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:00.437718727 +0000 UTC m=+34.763479628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls") pod "dns-default-8x4fw" (UID: "a94dabbc-b8ff-4236-9bdd-cc3b21d92c38") : secret "dns-default-metrics-tls" not found
Sep 10 20:58:59.937816 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.937737    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:59.937816 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:58:59.937793    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret podName:35386900-a40f-41d0-9c53-fe52d51e775a nodeName:}" failed. No retries permitted until 2026-09-10 20:59:31.937776909 +0000 UTC m=+66.263537809 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret") pod "global-pull-secret-syncer-dq9qc" (UID: "35386900-a40f-41d0-9c53-fe52d51e775a") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:59.937971 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.937946    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-config-volume\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:58:59.939934 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.939917    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/947f3e28-930c-467e-8de6-356a58785659-cert\") pod \"ingress-canary-q2w8b\" (UID: \"947f3e28-930c-467e-8de6-356a58785659\") " pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:58:59.951182 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.951159    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7psg\" (UniqueName: \"kubernetes.io/projected/947f3e28-930c-467e-8de6-356a58785659-kube-api-access-g7psg\") pod \"ingress-canary-q2w8b\" (UID: \"947f3e28-930c-467e-8de6-356a58785659\") " pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:58:59.956958 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:58:59.956938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnmbs\" (UniqueName: \"kubernetes.io/projected/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-kube-api-access-rnmbs\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:59:00.013865 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.013833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:00.038181 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.038154    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:59:00.038303 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.038281    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:59:00.038303 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.038294    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:59:00.038303 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.038303    2584 projected.go:196] Error preparing data for projected volume kube-api-access-hs8bb for pod openshift-network-diagnostics/network-check-target-pw4qs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:59:00.038412 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.038350    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb podName:7f0707f7-cc69-4ad2-95f2-95e4a8068ac8 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:32.038338377 +0000 UTC m=+66.364099289 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs8bb" (UniqueName: "kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb") pod "network-check-target-pw4qs" (UID: "7f0707f7-cc69-4ad2-95f2-95e4a8068ac8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:59:00.041110 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.041087    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2w8b"
Sep 10 20:59:00.173478 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.173245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:00.284560 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.284527    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:00.341511 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.341478    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:59:00.341849 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.341593    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:00.341849 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.341605    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dc64c7464-jzhls: secret "image-registry-tls" not found
Sep 10 20:59:00.341849 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.341652    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls podName:5c5445ae-52bb-4a45-a3e8-80b742b26b21 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:01.341639493 +0000 UTC m=+35.667400406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls") pod "image-registry-5dc64c7464-jzhls" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21") : secret "image-registry-tls" not found
Sep 10 20:59:00.361697 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.361669    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2w8b"]
Sep 10 20:59:00.365949 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:59:00.365924    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947f3e28_930c_467e_8de6_356a58785659.slice/crio-63e700fc6f20e238c154b9137c3e2f98ee79b857dbc6a257a5926c1bb22b7427 WatchSource:0}: Error finding container 63e700fc6f20e238c154b9137c3e2f98ee79b857dbc6a257a5926c1bb22b7427: Status 404 returned error can't find the container with id 63e700fc6f20e238c154b9137c3e2f98ee79b857dbc6a257a5926c1bb22b7427
Sep 10 20:59:00.398194 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.398164    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:00.442327 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.442253    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:59:00.442470 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.442374    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:00.442470 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:00.442444    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls podName:a94dabbc-b8ff-4236-9bdd-cc3b21d92c38 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:01.442412165 +0000 UTC m=+35.768173063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls") pod "dns-default-8x4fw" (UID: "a94dabbc-b8ff-4236-9bdd-cc3b21d92c38") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:00.513211 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.513172    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2w8b" event={"ID":"947f3e28-930c-467e-8de6-356a58785659","Type":"ContainerStarted","Data":"63e700fc6f20e238c154b9137c3e2f98ee79b857dbc6a257a5926c1bb22b7427"}
Sep 10 20:59:00.516165 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.516139    2584 generic.go:356] "Generic (PLEG): container finished" podID="30b5cecd-c292-4962-8043-843101bb70c5" containerID="bad38f02de5e5b161b1893547c0f2d0bd35e166e8b0f3809e3c25cd7ea7f7292" exitCode=0
Sep 10 20:59:00.516274 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:00.516205    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerDied","Data":"bad38f02de5e5b161b1893547c0f2d0bd35e166e8b0f3809e3c25cd7ea7f7292"}
Sep 10 20:59:01.295437 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.295389    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:59:01.295437 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.295428    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:59:01.295685 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.295431    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:59:01.299837 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.299812    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-947bp\""
Sep 10 20:59:01.299982 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.299853    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 20:59:01.300608 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.300586    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 20:59:01.300715 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.300629    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 20:59:01.301510 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.301489    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 20:59:01.301587 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.301559    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5qc4z\""
Sep 10 20:59:01.347733 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.347692    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:59:01.348350 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:01.347877    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:01.348350 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:01.347894    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dc64c7464-jzhls: secret "image-registry-tls" not found
Sep 10 20:59:01.348350 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:01.347949    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls podName:5c5445ae-52bb-4a45-a3e8-80b742b26b21 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:03.347932059 +0000 UTC m=+37.673692964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls") pod "image-registry-5dc64c7464-jzhls" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21") : secret "image-registry-tls" not found
Sep 10 20:59:01.448657 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.448614    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:59:01.448827 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:01.448802    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:01.448903 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:01.448890    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls podName:a94dabbc-b8ff-4236-9bdd-cc3b21d92c38 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:03.448873895 +0000 UTC m=+37.774634793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls") pod "dns-default-8x4fw" (UID: "a94dabbc-b8ff-4236-9bdd-cc3b21d92c38") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:01.521949 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.521909    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" event={"ID":"30b5cecd-c292-4962-8043-843101bb70c5","Type":"ContainerStarted","Data":"1d6150010e9baeab5be177185294e9d5ab2ddd1bbae877997717c2ba759171cb"}
Sep 10 20:59:01.561638 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.561548    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qlqrg" podStartSLOduration=8.144990339 podStartE2EDuration="35.561532592s" totalImagesPullingTime="27.416542253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:01.561283939 +0000 UTC m=+35.887044875" watchObservedRunningTime="2026-09-10 20:59:01.561532592 +0000 UTC m=+35.887293513"
Sep 10 20:59:01.787370 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.787333    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:01.997548 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:01.997469    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:02.127544 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:02.127507    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:02.879628 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:02.879589    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.264635 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.264609    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.365008 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.364971    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:59:03.365156 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:03.365113    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:03.365156 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:03.365130    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dc64c7464-jzhls: secret "image-registry-tls" not found
Sep 10 20:59:03.365220 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:03.365182    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls podName:5c5445ae-52bb-4a45-a3e8-80b742b26b21 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:07.365167926 +0000 UTC m=+41.690928839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls") pod "image-registry-5dc64c7464-jzhls" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21") : secret "image-registry-tls" not found
Sep 10 20:59:03.394545 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.394506    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.466113 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.466086    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:59:03.466276 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:03.466254    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:03.466338 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:03.466327    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls podName:a94dabbc-b8ff-4236-9bdd-cc3b21d92c38 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:07.466307886 +0000 UTC m=+41.792068808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls") pod "dns-default-8x4fw" (UID: "a94dabbc-b8ff-4236-9bdd-cc3b21d92c38") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:03.527800 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.527719    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2w8b" event={"ID":"947f3e28-930c-467e-8de6-356a58785659","Type":"ContainerStarted","Data":"54b17d4c819707b96c7cbd8c32bf3c7c84196a01a64215e30e8cda3ae78bddea"}
Sep 10 20:59:03.555044 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.554979    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q2w8b" podStartSLOduration=2.235490802 podStartE2EDuration="4.554964259s" totalImagesPullingTime="2.319473457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:03.554218203 +0000 UTC m=+37.879979125" watchObservedRunningTime="2026-09-10 20:59:03.554964259 +0000 UTC m=+37.880725179"
Sep 10 20:59:03.691993 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.691956    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.841308 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:03.841220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:04.395294 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:04.395263    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:04.777586 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:04.777498    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:04.911033 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:04.910994    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:05.255477 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:05.255444    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:07.396113 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:07.396076    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:59:07.396642 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:07.396257    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:07.396642 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:07.396288    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dc64c7464-jzhls: secret "image-registry-tls" not found
Sep 10 20:59:07.396642 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:07.396364    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls podName:5c5445ae-52bb-4a45-a3e8-80b742b26b21 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:15.396342841 +0000 UTC m=+49.722103756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls") pod "image-registry-5dc64c7464-jzhls" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21") : secret "image-registry-tls" not found
Sep 10 20:59:07.496803 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:07.496766    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:59:07.496957 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:07.496913    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:07.497004 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:07.496994    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls podName:a94dabbc-b8ff-4236-9bdd-cc3b21d92c38 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:15.496976143 +0000 UTC m=+49.822737042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls") pod "dns-default-8x4fw" (UID: "a94dabbc-b8ff-4236-9bdd-cc3b21d92c38") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:11.548889 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:11.548855    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:11.726324 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:11.726241    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:14.287138 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.287109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:14.414487 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.414445    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:14.949823 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.949789    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-596c47f5b-6k94l"]
Sep 10 20:59:14.994691 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.994666    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-596c47f5b-6k94l"]
Sep 10 20:59:14.994836 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.994758    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:14.998172 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.998135    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 20:59:14.998288 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.998176    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 20:59:14.998898 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.998884    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 20:59:14.998971 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.998955    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 20:59:14.999091 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.999078    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 20:59:14.999626 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.999610    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 20:59:14.999867 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:14.999830    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cz4hg\""
Sep 10 20:59:15.148285 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.148250    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-default-certificate\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.148285 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.148289    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpv9f\" (UniqueName: \"kubernetes.io/projected/314e11aa-c02f-4cad-a770-24604cebd511-kube-api-access-cpv9f\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.148521 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.148343    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-stats-auth\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.148521 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.148361    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.148521 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.148387    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.249692 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.249605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.249692 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.249669    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-default-certificate\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.249692 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.249691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpv9f\" (UniqueName: \"kubernetes.io/projected/314e11aa-c02f-4cad-a770-24604cebd511-kube-api-access-cpv9f\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.249897 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.249822    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:15.749796823 +0000 UTC m=+50.075557746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : configmap references non-existent config key: service-ca.crt
Sep 10 20:59:15.249897 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.249873    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-stats-auth\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.249967 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.249909    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.250062 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.250047    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 20:59:15.250111 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.250102    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:15.750089708 +0000 UTC m=+50.075850623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : secret "router-metrics-certs-default" not found
Sep 10 20:59:15.253021 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.252997    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-stats-auth\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.253125 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.253043    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-default-certificate\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.277241 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.277218    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpv9f\" (UniqueName: \"kubernetes.io/projected/314e11aa-c02f-4cad-a770-24604cebd511-kube-api-access-cpv9f\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.374385 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.374352    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:15.451689 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.451659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:59:15.451826 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.451807    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:15.451861 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.451830    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dc64c7464-jzhls: secret "image-registry-tls" not found
Sep 10 20:59:15.451890 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.451881    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls podName:5c5445ae-52bb-4a45-a3e8-80b742b26b21 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:31.451865728 +0000 UTC m=+65.777626627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls") pod "image-registry-5dc64c7464-jzhls" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21") : secret "image-registry-tls" not found
Sep 10 20:59:15.517145 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.517065    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:15.552267 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.552245    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:59:15.552411 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.552395    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:15.552473 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.552464    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls podName:a94dabbc-b8ff-4236-9bdd-cc3b21d92c38 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:31.552449407 +0000 UTC m=+65.878210306 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls") pod "dns-default-8x4fw" (UID: "a94dabbc-b8ff-4236-9bdd-cc3b21d92c38") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:15.618665 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.618632    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:15.753605 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.753568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.753800 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.753633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:15.753800 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.753734    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 20:59:15.753901 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.753839    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:16.753821281 +0000 UTC m=+51.079582179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : configmap references non-existent config key: service-ca.crt
Sep 10 20:59:15.753901 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:15.753864    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:16.7538576 +0000 UTC m=+51.079618498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : secret "router-metrics-certs-default" not found
Sep 10 20:59:15.979787 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:15.979752    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:16.101895 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:16.101862    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:16.761104 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:16.761060    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:16.761558 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:16.761134    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:16.761558 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:16.761210    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 20:59:16.761558 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:16.761280    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:18.761264732 +0000 UTC m=+53.087025631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : secret "router-metrics-certs-default" not found
Sep 10 20:59:16.761558 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:16.761295    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:18.761289378 +0000 UTC m=+53.087050277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : configmap references non-existent config key: service-ca.crt
Sep 10 20:59:18.777244 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:18.777207    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:18.777715 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:18.777255    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:18.777715 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:18.777356    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:22.77734354 +0000 UTC m=+57.103104438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : configmap references non-existent config key: service-ca.crt
Sep 10 20:59:18.777715 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:18.777354    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 20:59:18.777715 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:18.777406    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:22.777399139 +0000 UTC m=+57.103160051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : secret "router-metrics-certs-default" not found
Sep 10 20:59:19.048102 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:19.048064    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:19.164816 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:19.164775    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:20.330061 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:20.330021    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:20.412496 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:20.412461    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:20.995317 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:20.995282    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:21.076709 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.076665    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:21.253362 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.253275    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:21.261084 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.261057    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-96ph4"]
Sep 10 20:59:21.263024 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.263005    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.270827 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.270808    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 20:59:21.271670 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.271655    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 20:59:21.271958 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.271945    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 20:59:21.272565 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.272552    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7vxwx\""
Sep 10 20:59:21.272847 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.272832    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 20:59:21.286616 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.286595    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-96ph4"]
Sep 10 20:59:21.379818 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.379782    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:21.397800 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.397773    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dlsd\" (UniqueName: \"kubernetes.io/projected/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-api-access-6dlsd\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.397891 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.397813    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-crio-socket\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.397891 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.397885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-data-volume\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.397991 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.397903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.398043 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.398016    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.470228 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.470193    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:21.499008 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.498981    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlsd\" (UniqueName: \"kubernetes.io/projected/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-api-access-6dlsd\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.499115 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.499017    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-crio-socket\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.499115 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.499100    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-crio-socket\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.499182 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.499151    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-data-volume\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.499182 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.499172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.499252 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.499213    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.499303 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:21.499287    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:21.499303 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:21.499296    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:21.499381 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:21.499362    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:21.99934638 +0000 UTC m=+56.325107290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:21.499381 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:21.499376    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:21.999369946 +0000 UTC m=+56.325130844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:21.499535 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.499511    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-data-volume\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.508555 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.508503    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dlsd\" (UniqueName: \"kubernetes.io/projected/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-api-access-6dlsd\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:21.742938 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.742899    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:21.856744 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:21.856654    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:22.003310 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:22.003279    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:22.003460 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:22.003330    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:22.003460 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:22.003445    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:22.003539 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:22.003510    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:23.00349614 +0000 UTC m=+57.329257038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:22.003539 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:22.003450    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:22.003615 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:22.003576    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:23.003564444 +0000 UTC m=+57.329325362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:22.694623 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:22.694581    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:22.808768 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:22.808735    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:22.808938 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:22.808838    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:22.808938 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:22.808902    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:30.808883499 +0000 UTC m=+65.134644414 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : configmap references non-existent config key: service-ca.crt
Sep 10 20:59:22.808938 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:22.808926    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 20:59:22.809054 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:22.808972    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:30.808960307 +0000 UTC m=+65.134721220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : secret "router-metrics-certs-default" not found
Sep 10 20:59:22.828666 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:22.828615    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:22.978975 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:22.978911    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fz9wf_09521e42-9209-4a90-bead-2f4f4b6fdc18/dns-node-resolver/0.log"
Sep 10 20:59:23.009747 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:23.009715    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:23.009877 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:23.009809    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:23.009877 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:23.009829    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:23.009944 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:23.009898    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:23.009944 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:23.009906    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:25.009888939 +0000 UTC m=+59.335649856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:23.009944 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:23.009934    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:25.009924181 +0000 UTC m=+59.335685079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:23.448077 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:23.448038    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:23.688284 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:23.688237    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:24.374227 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:24.374198    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sqjpl_9f7f5cd3-168b-4020-a3be-79c741f85e9b/node-ca/0.log"
Sep 10 20:59:24.913512 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:24.913480    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:24.977024 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:24.976998    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q2w8b_947f3e28-930c-467e-8de6-356a58785659/serve-healthcheck-canary/0.log"
Sep 10 20:59:25.025312 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:25.025283    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:25.025445 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:25.025406    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:25.025489 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:25.025404    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:25.025489 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:25.025478    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:29.025462322 +0000 UTC m=+63.351223222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:25.025565 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:25.025500    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:25.025623 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:25.025612    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:29.025595259 +0000 UTC m=+63.351356172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:25.506807 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:25.506779    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cs6qr"
Sep 10 20:59:28.408576 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:28.408538    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:28.575464 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:28.575412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:28.703440 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:28.703337    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:28.972321 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:28.972234    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:29.058763 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:29.058732    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:29.058917 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:29.058786    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:29.058917 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:29.058867    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:29.058917 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:29.058875    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:29.059012 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:29.058928    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:37.058914551 +0000 UTC m=+71.384675450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:29.059012 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:29.058941    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:37.058935271 +0000 UTC m=+71.384696169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:29.073562 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:29.073525    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:29.649162 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:29.649130    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:30.101211 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:30.101133    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:30.655398 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:30.655364    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:30.870613 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:30.870576    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:30.870769 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:30.870630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:30.870769 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:30.870721    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 20:59:30.870769 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:30.870755    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:46.870742558 +0000 UTC m=+81.196503471 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : configmap references non-existent config key: service-ca.crt
Sep 10 20:59:30.870769 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:30.870770    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs podName:314e11aa-c02f-4cad-a770-24604cebd511 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:46.870763844 +0000 UTC m=+81.196524742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs") pod "router-default-596c47f5b-6k94l" (UID: "314e11aa-c02f-4cad-a770-24604cebd511") : secret "router-metrics-certs-default" not found
Sep 10 20:59:31.074484 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.074396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:31.456765 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.456731    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:31.474316 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.474289    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 20:59:31.474468 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:31.474451    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:31.474554 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:31.474470    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dc64c7464-jzhls: secret "image-registry-tls" not found
Sep 10 20:59:31.474554 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:31.474525    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls podName:5c5445ae-52bb-4a45-a3e8-80b742b26b21 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:03.47451238 +0000 UTC m=+97.800273279 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls") pod "image-registry-5dc64c7464-jzhls" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21") : secret "image-registry-tls" not found
Sep 10 20:59:31.574966 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.574932    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 20:59:31.575147 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:31.575120    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:31.575219 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:31.575203    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls podName:a94dabbc-b8ff-4236-9bdd-cc3b21d92c38 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:03.575182214 +0000 UTC m=+97.900943114 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls") pod "dns-default-8x4fw" (UID: "a94dabbc-b8ff-4236-9bdd-cc3b21d92c38") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:31.857474 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.857356    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:31.877560 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.877533    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 20:59:31.885858 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.885833    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 20:59:31.889003 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:31.888920    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 20:59:31.889353 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:31.889336    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs podName:2f4353e4-9d2e-438d-968d-85d5039afa54 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:35.889301741 +0000 UTC m=+130.215062653 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs") pod "network-metrics-daemon-4k2zp" (UID: "2f4353e4-9d2e-438d-968d-85d5039afa54") : secret "metrics-daemon-secret" not found
Sep 10 20:59:31.978487 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.978451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:59:31.981986 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.981966    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 20:59:31.991456 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:31.991435    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/35386900-a40f-41d0-9c53-fe52d51e775a-original-pull-secret\") pod \"global-pull-secret-syncer-dq9qc\" (UID: \"35386900-a40f-41d0-9c53-fe52d51e775a\") " pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:59:32.079824 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.079788    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:59:32.082181 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.082159    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 20:59:32.093531 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.093509    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 20:59:32.102833 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.102812    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs8bb\" (UniqueName: \"kubernetes.io/projected/7f0707f7-cc69-4ad2-95f2-95e4a8068ac8-kube-api-access-hs8bb\") pod \"network-check-target-pw4qs\" (UID: \"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8\") " pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:59:32.146554 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.146526    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:32.210648 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.210621    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-947bp\""
Sep 10 20:59:32.218344 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.218321    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:59:32.220507 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.220490    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dq9qc"
Sep 10 20:59:32.247019 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.246989    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:32.346358 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.346330    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:32.523395 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.523312    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pw4qs"]
Sep 10 20:59:32.526093 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:59:32.526061    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0707f7_cc69_4ad2_95f2_95e4a8068ac8.slice/crio-84a5993b6d5e75a9abb43ad809367ddba54c3a801d1738ea69056fffaa4770af WatchSource:0}: Error finding container 84a5993b6d5e75a9abb43ad809367ddba54c3a801d1738ea69056fffaa4770af: Status 404 returned error can't find the container with id 84a5993b6d5e75a9abb43ad809367ddba54c3a801d1738ea69056fffaa4770af
Sep 10 20:59:32.527853 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.527832    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dq9qc"]
Sep 10 20:59:32.529732 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.529703    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:32.530227 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:59:32.530207    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35386900_a40f_41d0_9c53_fe52d51e775a.slice/crio-e648ec57f9e3aa332854bd608fd5c32382863405da7c5d4b65d05d9fac000f5c WatchSource:0}: Error finding container e648ec57f9e3aa332854bd608fd5c32382863405da7c5d4b65d05d9fac000f5c: Status 404 returned error can't find the container with id e648ec57f9e3aa332854bd608fd5c32382863405da7c5d4b65d05d9fac000f5c
Sep 10 20:59:32.594579 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.594546    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dq9qc" event={"ID":"35386900-a40f-41d0-9c53-fe52d51e775a","Type":"ContainerStarted","Data":"e648ec57f9e3aa332854bd608fd5c32382863405da7c5d4b65d05d9fac000f5c"}
Sep 10 20:59:32.595648 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.595627    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pw4qs" event={"ID":"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8","Type":"ContainerStarted","Data":"84a5993b6d5e75a9abb43ad809367ddba54c3a801d1738ea69056fffaa4770af"}
Sep 10 20:59:32.846095 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.846008    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:32.947370 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:32.947331    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:33.452263 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:33.452213    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:34.062240 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:34.062173    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:34.260176 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:34.260140    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:34.366998 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:34.366808    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:34.531621 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:34.531520    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:34.694406 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:34.694368    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.656943 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:35.656894    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.873325 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:35.873293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:36.075061 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:36.074950    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:36.240107 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:36.240072    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:36.441873 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:36.441812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:37.094052 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.094018    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:37.118252 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.118228    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:37.118376 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.118274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:37.118468 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:37.118390    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:37.118527 ip-10-0-128-196 kubenswrapper[2584]: E0910 20:59:37.118491    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls podName:09e38a3f-35ed-49ee-a2f3-b565197ef3b9 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:53.118468909 +0000 UTC m=+87.444229824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-96ph4" (UID: "09e38a3f-35ed-49ee-a2f3-b565197ef3b9") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:37.118722 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.118703    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:37.610221 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.610185    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pw4qs" event={"ID":"7f0707f7-cc69-4ad2-95f2-95e4a8068ac8","Type":"ContainerStarted","Data":"1a5b12e45de5eb758a27f3a9468998a813444741adc02d3af391c16e19d7c836"}
Sep 10 20:59:37.610407 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.610376    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 20:59:37.611748 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.611720    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dq9qc" event={"ID":"35386900-a40f-41d0-9c53-fe52d51e775a","Type":"ContainerStarted","Data":"6898b25093e0433f16a0c3bc4ca3b1c5c012fcabb577e8f314ff6f0fc76dc3d3"}
Sep 10 20:59:37.655387 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.655333    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pw4qs" podStartSLOduration=67.145707533 podStartE2EDuration="1m11.655314874s" totalImagesPullingTime="4.509607341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:37.654361566 +0000 UTC m=+71.980122478" watchObservedRunningTime="2026-09-10 20:59:37.655314874 +0000 UTC m=+71.981075795"
Sep 10 20:59:37.669362 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.669329    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:37.675962 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.675920    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dq9qc" podStartSLOduration=67.165092006 podStartE2EDuration="1m11.675904798s" totalImagesPullingTime="4.510812792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:37.673740312 +0000 UTC m=+71.999501232" watchObservedRunningTime="2026-09-10 20:59:37.675904798 +0000 UTC m=+72.001665719"
Sep 10 20:59:37.861600 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:37.861504    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:38.062434 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:38.062381    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:38.473410 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:38.473375    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:38.855383 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:38.855291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:39.080198 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:39.080165    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:39.735520 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:39.735481    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:40.094326 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:40.094242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:40.207838 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:40.207808    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:40.320460 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:40.320413    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:41.423733 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:41.423703    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:41.620882 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:41.620848    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:41.724781 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:41.724701    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:41.854198 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:41.854159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:46.895264 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:46.895206    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:46.895264 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:46.895268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:46.895907 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:46.895886    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/314e11aa-c02f-4cad-a770-24604cebd511-service-ca-bundle\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:46.897608 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:46.897587    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/314e11aa-c02f-4cad-a770-24604cebd511-metrics-certs\") pod \"router-default-596c47f5b-6k94l\" (UID: \"314e11aa-c02f-4cad-a770-24604cebd511\") " pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:47.105745 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:47.105710    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cz4hg\""
Sep 10 20:59:47.114015 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:47.113990    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:47.395570 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:47.395320    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-596c47f5b-6k94l"]
Sep 10 20:59:47.398249 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:59:47.398218    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314e11aa_c02f_4cad_a770_24604cebd511.slice/crio-b28bffca8985b21ccf929f8c3191fdc4901b0ffb185fbd78bf1ba971a27a64f0 WatchSource:0}: Error finding container b28bffca8985b21ccf929f8c3191fdc4901b0ffb185fbd78bf1ba971a27a64f0: Status 404 returned error can't find the container with id b28bffca8985b21ccf929f8c3191fdc4901b0ffb185fbd78bf1ba971a27a64f0
Sep 10 20:59:47.638226 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:47.638192    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-596c47f5b-6k94l" event={"ID":"314e11aa-c02f-4cad-a770-24604cebd511","Type":"ContainerStarted","Data":"dc3dc25188762844c7e208629b6b1664131dde52b6c55cd7cd7cb1b5e5a55696"}
Sep 10 20:59:47.638382 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:47.638240    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-596c47f5b-6k94l" event={"ID":"314e11aa-c02f-4cad-a770-24604cebd511","Type":"ContainerStarted","Data":"b28bffca8985b21ccf929f8c3191fdc4901b0ffb185fbd78bf1ba971a27a64f0"}
Sep 10 20:59:48.114300 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:48.114263    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:48.116652 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:48.116619    2584 patch_prober.go:28] interesting pod/router-default-596c47f5b-6k94l container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Sep 10 20:59:48.116652 ip-10-0-128-196 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Sep 10 20:59:48.116652 ip-10-0-128-196 kubenswrapper[2584]: [+]process-running ok
Sep 10 20:59:48.116652 ip-10-0-128-196 kubenswrapper[2584]: healthz check failed
Sep 10 20:59:48.116848 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:48.116674    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-596c47f5b-6k94l" podUID="314e11aa-c02f-4cad-a770-24604cebd511" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 20:59:48.718583 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:48.718541    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:48.849125 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:48.849091    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:49.115628 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:49.115552    2584 patch_prober.go:28] interesting pod/router-default-596c47f5b-6k94l container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Sep 10 20:59:49.115628 ip-10-0-128-196 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Sep 10 20:59:49.115628 ip-10-0-128-196 kubenswrapper[2584]: [+]process-running ok
Sep 10 20:59:49.115628 ip-10-0-128-196 kubenswrapper[2584]: healthz check failed
Sep 10 20:59:49.116043 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:49.115626    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-596c47f5b-6k94l" podUID="314e11aa-c02f-4cad-a770-24604cebd511" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 20:59:50.116081 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:50.116042    2584 patch_prober.go:28] interesting pod/router-default-596c47f5b-6k94l container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Sep 10 20:59:50.116081 ip-10-0-128-196 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Sep 10 20:59:50.116081 ip-10-0-128-196 kubenswrapper[2584]: [+]process-running ok
Sep 10 20:59:50.116081 ip-10-0-128-196 kubenswrapper[2584]: healthz check failed
Sep 10 20:59:50.116543 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:50.116102    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-596c47f5b-6k94l" podUID="314e11aa-c02f-4cad-a770-24604cebd511" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 20:59:51.116229 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:51.116195    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:51.116752 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:51.116456    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:51.117394 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:51.117376    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-596c47f5b-6k94l"
Sep 10 20:59:51.152780 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:51.152728    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-596c47f5b-6k94l" podStartSLOduration=37.152709714 podStartE2EDuration="37.152709714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:47.67493982 +0000 UTC m=+82.000700741" watchObservedRunningTime="2026-09-10 20:59:51.152709714 +0000 UTC m=+85.478470635"
Sep 10 20:59:51.523350 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:51.523311    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:51.770883 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:51.770843    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:51.997080 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:51.997046    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:52.412712 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.412676    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"]
Sep 10 20:59:52.416676 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.416648    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.429953 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.429929    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 20:59:52.430110 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.430006    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 20:59:52.430296 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.430283    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 20:59:52.430857 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.430841    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 20:59:52.430902 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.430881    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 20:59:52.431027 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.431014    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 20:59:52.431096 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.431080    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 20:59:52.515071 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.515039    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"]
Sep 10 20:59:52.535717 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.535684    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-hub\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.535876 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.535723    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.535876 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.535743    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97fsv\" (UniqueName: \"kubernetes.io/projected/c9912a54-5871-40df-b86c-533477504581-kube-api-access-97fsv\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.535876 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.535763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9912a54-5871-40df-b86c-533477504581-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.535876 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.535856    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-ca\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.536022 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.535886    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.637184 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.637087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-ca\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.637184 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.637127    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.637184 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.637179    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-hub\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.637551 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.637197    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.637551 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.637293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97fsv\" (UniqueName: \"kubernetes.io/projected/c9912a54-5871-40df-b86c-533477504581-kube-api-access-97fsv\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.637551 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.637327    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9912a54-5871-40df-b86c-533477504581-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.638174 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.638143    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9912a54-5871-40df-b86c-533477504581-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.639694 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.639655    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-ca\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.639808 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.639790    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-hub\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.639898 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.639882    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.639960 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.639941    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c9912a54-5871-40df-b86c-533477504581-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.676326 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.676290    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97fsv\" (UniqueName: \"kubernetes.io/projected/c9912a54-5871-40df-b86c-533477504581-kube-api-access-97fsv\") pod \"cluster-proxy-proxy-agent-76d499f976-fddkr\" (UID: \"c9912a54-5871-40df-b86c-533477504581\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.743009 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.742975    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"
Sep 10 20:59:52.749548 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:52.749514    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.074147 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.074124    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr"]
Sep 10 20:59:53.075869 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:59:53.075840    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9912a54_5871_40df_b86c_533477504581.slice/crio-458597dd75bf4f553494412e715894b0822ad57f61c957031b3ba49bfb3d04e0 WatchSource:0}: Error finding container 458597dd75bf4f553494412e715894b0822ad57f61c957031b3ba49bfb3d04e0: Status 404 returned error can't find the container with id 458597dd75bf4f553494412e715894b0822ad57f61c957031b3ba49bfb3d04e0
Sep 10 20:59:53.141643 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.141608    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:53.143843 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.143825    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09e38a3f-35ed-49ee-a2f3-b565197ef3b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-96ph4\" (UID: \"09e38a3f-35ed-49ee-a2f3-b565197ef3b9\") " pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:53.146219 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.146190    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.295002 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.294961    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.392382 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.392296    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7vxwx\""
Sep 10 20:59:53.402409 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.402385    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-96ph4"
Sep 10 20:59:53.619503 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.619470    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.654726 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.654698    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr" event={"ID":"c9912a54-5871-40df-b86c-533477504581","Type":"ContainerStarted","Data":"458597dd75bf4f553494412e715894b0822ad57f61c957031b3ba49bfb3d04e0"}
Sep 10 20:59:53.718273 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.718237    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-96ph4"]
Sep 10 20:59:53.724306 ip-10-0-128-196 kubenswrapper[2584]: W0910 20:59:53.724258    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e38a3f_35ed_49ee_a2f3_b565197ef3b9.slice/crio-0a752865136f1bf6f67695d61adf9eb534bb97fc9b42f1c5299623a79570e296 WatchSource:0}: Error finding container 0a752865136f1bf6f67695d61adf9eb534bb97fc9b42f1c5299623a79570e296: Status 404 returned error can't find the container with id 0a752865136f1bf6f67695d61adf9eb534bb97fc9b42f1c5299623a79570e296
Sep 10 20:59:53.839898 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:53.839862    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:54.359314 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:54.359272    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:54.660899 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:54.660866    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96ph4" event={"ID":"09e38a3f-35ed-49ee-a2f3-b565197ef3b9","Type":"ContainerStarted","Data":"0f5d87c7da4277daf4376581406d5b5cd2bb5dad1ddfffeefdb2f3fa3e0985d4"}
Sep 10 20:59:54.661330 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:54.660930    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96ph4" event={"ID":"09e38a3f-35ed-49ee-a2f3-b565197ef3b9","Type":"ContainerStarted","Data":"0a752865136f1bf6f67695d61adf9eb534bb97fc9b42f1c5299623a79570e296"}
Sep 10 20:59:55.665802 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:55.665761    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96ph4" event={"ID":"09e38a3f-35ed-49ee-a2f3-b565197ef3b9","Type":"ContainerStarted","Data":"46752de9da7dd8cbf1b8e46e8b34a8fed361f649eea87b22a11257646945ad4d"}
Sep 10 20:59:55.808787 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:55.808740    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:55.948141 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:55.948062    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:56.672268 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:56.672228    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr" event={"ID":"c9912a54-5871-40df-b86c-533477504581","Type":"ContainerStarted","Data":"43642fad63b9f46f14c03bfe19607086b1f0d2bd6d0a94c8bde1a891ccc0462a"}
Sep 10 20:59:56.672718 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:56.672303    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr" event={"ID":"c9912a54-5871-40df-b86c-533477504581","Type":"ContainerStarted","Data":"03d00b64e28a68972da0adf53d1a89af989459729ca1fd3fca2775f4ac5b02d6"}
Sep 10 20:59:56.755393 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:56.755343    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:57.677150 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:57.677121    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr" event={"ID":"c9912a54-5871-40df-b86c-533477504581","Type":"ContainerStarted","Data":"8197c2a22d75731e84a87e07dbc76238879d1f471e82a20eaadbe8ae5a5ec171"}
Sep 10 20:59:57.679354 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:57.679326    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-96ph4" event={"ID":"09e38a3f-35ed-49ee-a2f3-b565197ef3b9","Type":"ContainerStarted","Data":"552c1df61a4c39285f0c4140c3e6492804078b645495f9cae06654086740fe7b"}
Sep 10 20:59:57.703739 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:57.703672    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76d499f976-fddkr" podStartSLOduration=2.267459643 podStartE2EDuration="5.703657348s" totalImagesPullingTime="3.436197705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:57.703500341 +0000 UTC m=+92.029261263" watchObservedRunningTime="2026-09-10 20:59:57.703657348 +0000 UTC m=+92.029418281"
Sep 10 20:59:57.733139 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:57.733080    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-96ph4" podStartSLOduration=33.246224216 podStartE2EDuration="36.733062367s" totalImagesPullingTime="3.486838151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:59:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:57.732230679 +0000 UTC m=+92.057991600" watchObservedRunningTime="2026-09-10 20:59:57.733062367 +0000 UTC m=+92.058823288"
Sep 10 20:59:58.927057 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:58.927018    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:59.160146 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:59.160111    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:59.378571 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:59.378467    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:59.642809 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:59.642715    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:59.756008 ip-10-0-128-196 kubenswrapper[2584]: I0910 20:59:59.755969    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:00.157740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:00.157696    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:00.314947 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:00.314916    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:01.487024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:01.486985    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:01.644263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:01.644231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:02.387166 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.387127    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:02.522280 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.522247    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9kl4x"]
Sep 10 21:00:02.525652 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.525629    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.527905 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.527874    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 21:00:02.528019 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.528010    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 21:00:02.530867 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.530846    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 21:00:02.531128 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.531107    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 21:00:02.531276 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.531260    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 21:00:02.531650 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.531634    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 21:00:02.538203 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.538181    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m26fb\""
Sep 10 21:00:02.571743 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.571703    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:02.613703 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.613664    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-tls\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.613891 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.613727    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-textfile\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.613891 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.613779    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-sys\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.613891 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.613861    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.614071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.613902    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85fb2084-2834-4cdb-aa5c-b077c5c381bc-metrics-client-ca\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.614071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.613985    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.614071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.614027    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-wtmp\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.614071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.614050    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-root\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.614071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.614067    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2bq2\" (UniqueName: \"kubernetes.io/projected/85fb2084-2834-4cdb-aa5c-b077c5c381bc-kube-api-access-s2bq2\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715276 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715233    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715276 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715279    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-wtmp\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-root\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715328    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2bq2\" (UniqueName: \"kubernetes.io/projected/85fb2084-2834-4cdb-aa5c-b077c5c381bc-kube-api-access-s2bq2\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715349    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-tls\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715383    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-textfile\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715392    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-root\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715446    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-sys\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715494    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-wtmp\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715563 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715566    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85fb2084-2834-4cdb-aa5c-b077c5c381bc-metrics-client-ca\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715902 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715579    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85fb2084-2834-4cdb-aa5c-b077c5c381bc-sys\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.715985 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715964    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.716094 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.715983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-textfile\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.716134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.716104    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85fb2084-2834-4cdb-aa5c-b077c5c381bc-metrics-client-ca\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.717939 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.717916    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-tls\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.718032 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.717950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85fb2084-2834-4cdb-aa5c-b077c5c381bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.742120 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.742088    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2bq2\" (UniqueName: \"kubernetes.io/projected/85fb2084-2834-4cdb-aa5c-b077c5c381bc-kube-api-access-s2bq2\") pod \"node-exporter-9kl4x\" (UID: \"85fb2084-2834-4cdb-aa5c-b077c5c381bc\") " pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.808331 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.808290    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:02.834669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:02.834640    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9kl4x"
Sep 10 21:00:02.847624 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:00:02.847561    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fb2084_2834_4cdb_aa5c_b077c5c381bc.slice/crio-5cc57eb77db5422863a5c0aba8e678ad45a5337552721512fc89a2b349c69dff WatchSource:0}: Error finding container 5cc57eb77db5422863a5c0aba8e678ad45a5337552721512fc89a2b349c69dff: Status 404 returned error can't find the container with id 5cc57eb77db5422863a5c0aba8e678ad45a5337552721512fc89a2b349c69dff
Sep 10 21:00:03.025785 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.025667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:03.522223 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.522184    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 21:00:03.524649 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.524628    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"image-registry-5dc64c7464-jzhls\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") " pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 21:00:03.548155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.548122    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bbgkn\""
Sep 10 21:00:03.556372 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.556342    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 21:00:03.623308 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.623278    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 21:00:03.625594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.625565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a94dabbc-b8ff-4236-9bdd-cc3b21d92c38-metrics-tls\") pod \"dns-default-8x4fw\" (UID: \"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38\") " pod="openshift-dns/dns-default-8x4fw"
Sep 10 21:00:03.641438 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.641404    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vl2hm\""
Sep 10 21:00:03.646810 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.646795    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8x4fw"
Sep 10 21:00:03.678651 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.678602    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:03.699126 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.699091    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9kl4x" event={"ID":"85fb2084-2834-4cdb-aa5c-b077c5c381bc","Type":"ContainerStarted","Data":"5cc57eb77db5422863a5c0aba8e678ad45a5337552721512fc89a2b349c69dff"}
Sep 10 21:00:03.841577 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.841492    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dc64c7464-jzhls"]
Sep 10 21:00:03.845051 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:00:03.845013    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c5445ae_52bb_4a45_a3e8_80b742b26b21.slice/crio-43fea07250c9554ddd9d3a13de942d3237242faefab613e71f6279dab45b7c5b WatchSource:0}: Error finding container 43fea07250c9554ddd9d3a13de942d3237242faefab613e71f6279dab45b7c5b: Status 404 returned error can't find the container with id 43fea07250c9554ddd9d3a13de942d3237242faefab613e71f6279dab45b7c5b
Sep 10 21:00:03.849384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.849363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:03.922697 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.922662    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8x4fw"]
Sep 10 21:00:03.926908 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:00:03.926882    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94dabbc_b8ff_4236_9bdd_cc3b21d92c38.slice/crio-3b9215000b2e38f5fda03833e5d0ec84e3f69de4a1f6b8851fc5a32fab7e61aa WatchSource:0}: Error finding container 3b9215000b2e38f5fda03833e5d0ec84e3f69de4a1f6b8851fc5a32fab7e61aa: Status 404 returned error can't find the container with id 3b9215000b2e38f5fda03833e5d0ec84e3f69de4a1f6b8851fc5a32fab7e61aa
Sep 10 21:00:03.937100 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:03.937073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:04.704849 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:04.704797    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls" event={"ID":"5c5445ae-52bb-4a45-a3e8-80b742b26b21","Type":"ContainerStarted","Data":"66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0"}
Sep 10 21:00:04.705327 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:04.704854    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls" event={"ID":"5c5445ae-52bb-4a45-a3e8-80b742b26b21","Type":"ContainerStarted","Data":"43fea07250c9554ddd9d3a13de942d3237242faefab613e71f6279dab45b7c5b"}
Sep 10 21:00:04.705327 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:04.704880    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 21:00:04.706115 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:04.706085    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8x4fw" event={"ID":"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38","Type":"ContainerStarted","Data":"3b9215000b2e38f5fda03833e5d0ec84e3f69de4a1f6b8851fc5a32fab7e61aa"}
Sep 10 21:00:05.675468 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:05.675412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:05.711961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:05.711925    2584 generic.go:356] "Generic (PLEG): container finished" podID="85fb2084-2834-4cdb-aa5c-b077c5c381bc" containerID="8629a7c0e5a932adced0883ea7dc87b37a565b9b12b669de43b99530559ab2d3" exitCode=0
Sep 10 21:00:05.712433 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:05.712010    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9kl4x" event={"ID":"85fb2084-2834-4cdb-aa5c-b077c5c381bc","Type":"ContainerDied","Data":"8629a7c0e5a932adced0883ea7dc87b37a565b9b12b669de43b99530559ab2d3"}
Sep 10 21:00:05.758883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:05.758828    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls" podStartSLOduration=99.758809634 podStartE2EDuration="1m39.758809634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:04.738222158 +0000 UTC m=+99.063983079" watchObservedRunningTime="2026-09-10 21:00:05.758809634 +0000 UTC m=+100.084570554"
Sep 10 21:00:05.825228 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:05.825194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:05.952221 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:05.952133    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:06.719840 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.719802    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9kl4x" event={"ID":"85fb2084-2834-4cdb-aa5c-b077c5c381bc","Type":"ContainerStarted","Data":"f63022d807f805cc844958770d07a8c5c2f47b35509ea1fc45c3e9f796f81562"}
Sep 10 21:00:06.720297 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.719881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9kl4x" event={"ID":"85fb2084-2834-4cdb-aa5c-b077c5c381bc","Type":"ContainerStarted","Data":"48c2bc242aa8de4a2b92cbb49d9c2c7bb02e729db0ea0a2c0e2c2239ec8581be"}
Sep 10 21:00:06.965982 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.965930    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9kl4x" podStartSLOduration=2.785840736 podStartE2EDuration="4.96591212s" totalImagesPullingTime="2.118307848s" totalInitContainerRuntime="61.763536ms" isStatefulPod=true podCreationTimestamp="2026-09-10 21:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:06.748030803 +0000 UTC m=+101.073791726" watchObservedRunningTime="2026-09-10 21:00:06.96591212 +0000 UTC m=+101.291673032"
Sep 10 21:00:06.966392 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.966372    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-d446cbf8b-wfblv"]
Sep 10 21:00:06.969372 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.969349    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:06.972530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.972474    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 21:00:06.972637 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.972604    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 21:00:06.972876 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.972850    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 21:00:06.972984 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.972947    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 21:00:06.973109 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.973066    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dvoua72ep3k5o\""
Sep 10 21:00:06.973800 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.973781    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6hrjq\""
Sep 10 21:00:06.993779 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:06.993756    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d446cbf8b-wfblv"]
Sep 10 21:00:07.052303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.052265    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-client-ca-bundle\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.052303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.052302    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/237b84a4-64bb-452f-a55e-42023f356222-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.052572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.052346    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/237b84a4-64bb-452f-a55e-42023f356222-metrics-server-audit-profiles\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.052572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.052442    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c292\" (UniqueName: \"kubernetes.io/projected/237b84a4-64bb-452f-a55e-42023f356222-kube-api-access-9c292\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.052572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.052489    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/237b84a4-64bb-452f-a55e-42023f356222-audit-log\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.052572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.052519    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-secret-metrics-server-client-certs\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.052572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.052558    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-secret-metrics-server-tls\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.148624 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.148582    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:07.153468 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.153449    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-client-ca-bundle\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.153524 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.153489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/237b84a4-64bb-452f-a55e-42023f356222-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.153565 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.153528    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/237b84a4-64bb-452f-a55e-42023f356222-metrics-server-audit-profiles\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.153565 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.153547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c292\" (UniqueName: \"kubernetes.io/projected/237b84a4-64bb-452f-a55e-42023f356222-kube-api-access-9c292\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.153662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.153569    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/237b84a4-64bb-452f-a55e-42023f356222-audit-log\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.153662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.153587    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-secret-metrics-server-client-certs\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.153662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.153606    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-secret-metrics-server-tls\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.154047 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.154024    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/237b84a4-64bb-452f-a55e-42023f356222-audit-log\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.154408 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.154386    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/237b84a4-64bb-452f-a55e-42023f356222-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.154766 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.154735    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/237b84a4-64bb-452f-a55e-42023f356222-metrics-server-audit-profiles\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.156069 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.156050    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-secret-metrics-server-client-certs\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.156155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.156120    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-secret-metrics-server-tls\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.156305 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.156287    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/237b84a4-64bb-452f-a55e-42023f356222-client-ca-bundle\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.178518 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.178484    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c292\" (UniqueName: \"kubernetes.io/projected/237b84a4-64bb-452f-a55e-42023f356222-kube-api-access-9c292\") pod \"metrics-server-d446cbf8b-wfblv\" (UID: \"237b84a4-64bb-452f-a55e-42023f356222\") " pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.279035 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.278944    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:07.286629 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.286601    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f58bc88fb-4xzlv"]
Sep 10 21:00:07.289881 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.289855    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.300748 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.300713    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dtlm8\""
Sep 10 21:00:07.300748 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.300737    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 21:00:07.300927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.300713    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 21:00:07.300927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.300886    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 21:00:07.301027 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.300945    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 21:00:07.301027 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.300997    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 21:00:07.301182 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.301161    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 21:00:07.325533 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.325502    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 21:00:07.332518 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.332395    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 21:00:07.354809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.354780    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-service-ca\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.355034 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.354864    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-trusted-ca-bundle\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.355078 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.355065    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-oauth-config\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.355112 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.355105    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-serving-cert\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.355196 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.355184    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9jt7\" (UniqueName: \"kubernetes.io/projected/fd89fd26-d84b-4630-a841-fc1262604ba0-kube-api-access-v9jt7\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.355264 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.355247    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-console-config\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.355305 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.355292    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-oauth-serving-cert\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.398306 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.398267    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:07.412782 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.412743    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f58bc88fb-4xzlv"]
Sep 10 21:00:07.456523 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.456488    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-trusted-ca-bundle\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.456731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.456531    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-oauth-config\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.456731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.456566    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-serving-cert\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.456731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.456612    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9jt7\" (UniqueName: \"kubernetes.io/projected/fd89fd26-d84b-4630-a841-fc1262604ba0-kube-api-access-v9jt7\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.456731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.456636    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-console-config\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.456731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.456651    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-oauth-serving-cert\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.456731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.456674    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-service-ca\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.457522 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.457493    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-trusted-ca-bundle\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.457522 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.457495    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-console-config\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.457653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.457524    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-oauth-serving-cert\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.457653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.457531    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-service-ca\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.459004 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.458981    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-oauth-config\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.459192 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.459174    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-serving-cert\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.476521 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.476491    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9jt7\" (UniqueName: \"kubernetes.io/projected/fd89fd26-d84b-4630-a841-fc1262604ba0-kube-api-access-v9jt7\") pod \"console-6f58bc88fb-4xzlv\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") " pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.564553 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.564453    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d446cbf8b-wfblv"]
Sep 10 21:00:07.570838 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:00:07.570813    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237b84a4_64bb_452f_a55e_42023f356222.slice/crio-cd24e309a2462166a825ee99ef1d596251efefcb7c05ef2e958f5e81f49e0060 WatchSource:0}: Error finding container cd24e309a2462166a825ee99ef1d596251efefcb7c05ef2e958f5e81f49e0060: Status 404 returned error can't find the container with id cd24e309a2462166a825ee99ef1d596251efefcb7c05ef2e958f5e81f49e0060
Sep 10 21:00:07.600085 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.600056    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:07.658737 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.658705    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:07.724854 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.724802    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8x4fw" event={"ID":"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38","Type":"ContainerStarted","Data":"ccc9a156d7af8d0ca0f088289cfe88ffd175e4617a20dc701c362931d2fb932b"}
Sep 10 21:00:07.725282 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.724881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8x4fw" event={"ID":"a94dabbc-b8ff-4236-9bdd-cc3b21d92c38","Type":"ContainerStarted","Data":"e8ada3d430917abd4455ddcbaac803f0ba3afe77f994b2647a1893248a124ea2"}
Sep 10 21:00:07.725282 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.725014    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8x4fw"
Sep 10 21:00:07.726103 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.726082    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv" event={"ID":"237b84a4-64bb-452f-a55e-42023f356222","Type":"ContainerStarted","Data":"cd24e309a2462166a825ee99ef1d596251efefcb7c05ef2e958f5e81f49e0060"}
Sep 10 21:00:07.775791 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.775725    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8x4fw" podStartSLOduration=65.933847514 podStartE2EDuration="1m8.775695823s" totalImagesPullingTime="2.841848309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:07.770533663 +0000 UTC m=+102.096294584" watchObservedRunningTime="2026-09-10 21:00:07.775695823 +0000 UTC m=+102.101456745"
Sep 10 21:00:07.900174 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:07.900146    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f58bc88fb-4xzlv"]
Sep 10 21:00:07.900965 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:00:07.900930    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd89fd26_d84b_4630_a841_fc1262604ba0.slice/crio-6899ccdf827804899dccae30010fb1cc9f8878e875a6b3892fc3ed5285992595 WatchSource:0}: Error finding container 6899ccdf827804899dccae30010fb1cc9f8878e875a6b3892fc3ed5285992595: Status 404 returned error can't find the container with id 6899ccdf827804899dccae30010fb1cc9f8878e875a6b3892fc3ed5285992595
Sep 10 21:00:08.352930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.352836    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:08.455178 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.455146    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:08.616920 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.616845    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pw4qs"
Sep 10 21:00:08.734021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.733978    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f58bc88fb-4xzlv" event={"ID":"fd89fd26-d84b-4630-a841-fc1262604ba0","Type":"ContainerStarted","Data":"6899ccdf827804899dccae30010fb1cc9f8878e875a6b3892fc3ed5285992595"}
Sep 10 21:00:08.836982 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.836948    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:00:08.845672 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.845644    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.847746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.847683    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:08.851324 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.851178    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 21:00:08.851324 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.851226    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 21:00:08.851324 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.851178    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 21:00:08.851324 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.851233    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 21:00:08.851324 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.851231    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 21:00:08.852214 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.852191    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 21:00:08.853370 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.853219    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 21:00:08.853370 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.853298    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 21:00:08.853585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.853452    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 21:00:08.853585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.853502    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 21:00:08.853733 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.853588    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 21:00:08.853789 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.853772    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 21:00:08.853984 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.853919    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-10c0tp8k75j3r\""
Sep 10 21:00:08.857407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.857385    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-t5frq\""
Sep 10 21:00:08.858293 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.858270    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 21:00:08.861571 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.861548    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 21:00:08.894245 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.894165    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:00:08.969941 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.969901    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdq6l\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-kube-api-access-hdq6l\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.969962    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970004    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970028    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970065    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970116    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970455 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970184    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970455 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970256    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970455 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970290    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970455 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970373    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970455 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970406    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970466    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970501    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970544    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970572    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970600    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970625    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.970719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970694    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.971034 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.970748    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-config\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:08.980737 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:08.980696    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:09.071414 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071379    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.071607 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071439    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.071607 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071467    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.071607 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.071607 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071516    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.071607 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071543    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.071607 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071580    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.071913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071900    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072022 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.071955    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072077 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072025    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072083    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072185 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072238 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072218    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072290 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072254    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072338 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072389 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072336    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-config\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072389 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072363    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdq6l\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-kube-api-access-hdq6l\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072511 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072403    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072511 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072462    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072511 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072487    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072649    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.072756 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072727    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.073161 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.073136    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.073263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.072084    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.073843 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.073816    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.074461 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.074436    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.074919 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.074604    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.074919 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.074804    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.075078 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.075016    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.075383 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.075360    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.075804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.075777    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.075804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.075791    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.077477 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.077453    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.077787 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.077749    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.077927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.077879    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.078072 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.078051    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.078385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.078366    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-config\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.079040 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.079020    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.079765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.079747    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.087412 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.087391    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdq6l\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-kube-api-access-hdq6l\") pod \"prometheus-k8s-0\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.160505 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.160468    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:09.474497 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.474467    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:00:09.477642 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:00:09.477613    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04728475_3abb_43b2_bd79_4b832220aa4c.slice/crio-d6e72a7417869d15b7815e676631a907a7afb35c5d68f1b472616360572e9c21 WatchSource:0}: Error finding container d6e72a7417869d15b7815e676631a907a7afb35c5d68f1b472616360572e9c21: Status 404 returned error can't find the container with id d6e72a7417869d15b7815e676631a907a7afb35c5d68f1b472616360572e9c21
Sep 10 21:00:09.738893 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.738808    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerStarted","Data":"d6e72a7417869d15b7815e676631a907a7afb35c5d68f1b472616360572e9c21"}
Sep 10 21:00:09.740359 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.740333    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv" event={"ID":"237b84a4-64bb-452f-a55e-42023f356222","Type":"ContainerStarted","Data":"8dc59b820dc8070d60eab0af75de9b8014af69bdadb654c0e49ba64905da9b77"}
Sep 10 21:00:09.777805 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:09.777745    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv" podStartSLOduration=1.990213905 podStartE2EDuration="3.777724711s" totalImagesPullingTime="1.787510806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:09.777186646 +0000 UTC m=+104.102947573" watchObservedRunningTime="2026-09-10 21:00:09.777724711 +0000 UTC m=+104.103485633"
Sep 10 21:00:11.268550 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:11.268522    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:11.428730 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:11.428658    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:11.639308 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:11.639217    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:11.749348 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:11.749289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f58bc88fb-4xzlv" event={"ID":"fd89fd26-d84b-4630-a841-fc1262604ba0","Type":"ContainerStarted","Data":"28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f"}
Sep 10 21:00:11.776964 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:11.776783    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f58bc88fb-4xzlv" podStartSLOduration=1.412768462 podStartE2EDuration="4.776762543s" totalImagesPullingTime="3.363994081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:11.776556376 +0000 UTC m=+106.102317309" watchObservedRunningTime="2026-09-10 21:00:11.776762543 +0000 UTC m=+106.102523466"
Sep 10 21:00:12.296948 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:12.296910    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:12.421270 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:12.421236    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:12.754321 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:12.754288    2584 generic.go:356] "Generic (PLEG): container finished" podID="04728475-3abb-43b2-bd79-4b832220aa4c" containerID="3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8" exitCode=0
Sep 10 21:00:12.754514 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:12.754372    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8"}
Sep 10 21:00:15.032583 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:15.032550    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dc64c7464-jzhls"]
Sep 10 21:00:15.321050 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:15.320964    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:15.450181 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:15.450125    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:15.726690 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:15.726635    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:15.864755 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:15.864720    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:16.358766 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:16.358732    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:16.417302 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:16.417259    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f58bc88fb-4xzlv"]
Sep 10 21:00:16.478399 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:16.478357    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:16.771180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:16.771094    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerStarted","Data":"21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0"}
Sep 10 21:00:16.771180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:16.771160    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerStarted","Data":"05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0"}
Sep 10 21:00:17.116258 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:17.116168    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:17.300147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:17.300106    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:17.600200 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:17.600168    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:17.737129 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:17.737090    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8x4fw"
Sep 10 21:00:18.781702 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:18.781666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerStarted","Data":"5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213"}
Sep 10 21:00:18.782206 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:18.781768    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerStarted","Data":"5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1"}
Sep 10 21:00:18.782206 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:18.781821    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerStarted","Data":"19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855"}
Sep 10 21:00:18.782206 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:18.781865    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerStarted","Data":"12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f"}
Sep 10 21:00:18.828146 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:18.828086    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.146914293 podStartE2EDuration="10.82806829s" totalImagesPullingTime="7.662571354s" totalInitContainerRuntime="18.582643ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:18.824673043 +0000 UTC m=+113.150433975" watchObservedRunningTime="2026-09-10 21:00:18.82806829 +0000 UTC m=+113.153829210"
Sep 10 21:00:19.161382 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:19.161353    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:00:21.489007 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:21.488968    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:21.721409 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:21.721369    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:25.037841 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:25.037812    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 21:00:26.915955 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:26.915898    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:27.062973 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:27.062938    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:27.279736 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:27.279646    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:27.279911 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:27.279752    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:27.900267 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:27.900233    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:28.064207 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:28.064176    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:28.232498 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:28.232394    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:33.669652 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:33.669608    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:33.765528 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:33.765490    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:35.915498 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:35.915462    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 21:00:35.917724 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:35.917701    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f4353e4-9d2e-438d-968d-85d5039afa54-metrics-certs\") pod \"network-metrics-daemon-4k2zp\" (UID: \"2f4353e4-9d2e-438d-968d-85d5039afa54\") " pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 21:00:36.118414 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:36.118383    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5qc4z\""
Sep 10 21:00:36.125860 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:36.125841    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4k2zp"
Sep 10 21:00:36.416446 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:36.416399    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4k2zp"]
Sep 10 21:00:36.419829 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:00:36.419789    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4353e4_9d2e_438d_968d_85d5039afa54.slice/crio-fe76102c5488b91acca573b520791eef7820b8ae46dbc127609d9b410b54e95c WatchSource:0}: Error finding container fe76102c5488b91acca573b520791eef7820b8ae46dbc127609d9b410b54e95c: Status 404 returned error can't find the container with id fe76102c5488b91acca573b520791eef7820b8ae46dbc127609d9b410b54e95c
Sep 10 21:00:36.844574 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:36.844536    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4k2zp" event={"ID":"2f4353e4-9d2e-438d-968d-85d5039afa54","Type":"ContainerStarted","Data":"fe76102c5488b91acca573b520791eef7820b8ae46dbc127609d9b410b54e95c"}
Sep 10 21:00:37.849722 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:37.849627    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4k2zp" event={"ID":"2f4353e4-9d2e-438d-968d-85d5039afa54","Type":"ContainerStarted","Data":"281adb07fbc3203a75b32a021bdc1446d6ffaeb7bc7830014791a42681645837"}
Sep 10 21:00:37.849722 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:37.849686    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4k2zp" event={"ID":"2f4353e4-9d2e-438d-968d-85d5039afa54","Type":"ContainerStarted","Data":"7c35d381ad9f359336f2d4b24ff54fa09f2c5795e930253f1b534823c9b4b828"}
Sep 10 21:00:37.874280 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:37.874220    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4k2zp" podStartSLOduration=130.770198417 podStartE2EDuration="2m11.874204476s" totalImagesPullingTime="1.104006059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:37.873069834 +0000 UTC m=+132.198830780" watchObservedRunningTime="2026-09-10 21:00:37.874204476 +0000 UTC m=+132.199965397"
Sep 10 21:00:38.104221 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:38.104133    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:38.237669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:38.237636    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:40.053258 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.053211    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls" podUID="5c5445ae-52bb-4a45-a3e8-80b742b26b21" containerName="registry" containerID="cri-o://66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0" gracePeriod=30
Sep 10 21:00:40.294769 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.294747    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 21:00:40.351968 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.351883    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg24n\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-kube-api-access-fg24n\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.351968 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.351927    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-bound-sa-token\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.352184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.351977    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-installation-pull-secrets\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.352184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.352029    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.352184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.352064    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-certificates\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.352184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.352110    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-trusted-ca\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.352184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.352144    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c5445ae-52bb-4a45-a3e8-80b742b26b21-ca-trust-extracted\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.352483 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.352192    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-image-registry-private-configuration\") pod \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\" (UID: \"5c5445ae-52bb-4a45-a3e8-80b742b26b21\") "
Sep 10 21:00:40.353284 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.353219    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:40.353431 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.353385    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:40.354710 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.354666    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:40.354710 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.354682    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-kube-api-access-fg24n" (OuterVolumeSpecName: "kube-api-access-fg24n") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "kube-api-access-fg24n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:40.354910 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.354886    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:40.355088 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.355057    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:40.355163 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.355080    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:40.361034 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.361008    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c5445ae-52bb-4a45-a3e8-80b742b26b21-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5c5445ae-52bb-4a45-a3e8-80b742b26b21" (UID: "5c5445ae-52bb-4a45-a3e8-80b742b26b21"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:00:40.453577 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453548    2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.453577 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453575    2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-registry-certificates\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.453732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453586    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c5445ae-52bb-4a45-a3e8-80b742b26b21-trusted-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.453732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453596    2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c5445ae-52bb-4a45-a3e8-80b742b26b21-ca-trust-extracted\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.453732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453608    2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-image-registry-private-configuration\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.453732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453617    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg24n\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-kube-api-access-fg24n\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.453732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453626    2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c5445ae-52bb-4a45-a3e8-80b742b26b21-bound-sa-token\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.453732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.453634    2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c5445ae-52bb-4a45-a3e8-80b742b26b21-installation-pull-secrets\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:40.861842 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.861807    2584 generic.go:356] "Generic (PLEG): container finished" podID="5c5445ae-52bb-4a45-a3e8-80b742b26b21" containerID="66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0" exitCode=0
Sep 10 21:00:40.862005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.861852    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls"
Sep 10 21:00:40.862005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.861860    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls" event={"ID":"5c5445ae-52bb-4a45-a3e8-80b742b26b21","Type":"ContainerDied","Data":"66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0"}
Sep 10 21:00:40.862005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.861902    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dc64c7464-jzhls" event={"ID":"5c5445ae-52bb-4a45-a3e8-80b742b26b21","Type":"ContainerDied","Data":"43fea07250c9554ddd9d3a13de942d3237242faefab613e71f6279dab45b7c5b"}
Sep 10 21:00:40.862005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.861921    2584 scope.go:117] "RemoveContainer" containerID="66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0"
Sep 10 21:00:40.892183 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.892157    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dc64c7464-jzhls"]
Sep 10 21:00:40.894976 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.894948    2584 scope.go:117] "RemoveContainer" containerID="66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0"
Sep 10 21:00:40.895291 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:00:40.895266    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0\": container with ID starting with 66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0 not found: ID does not exist" containerID="66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0"
Sep 10 21:00:40.895369 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.895302    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0"} err="failed to get container status \"66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0\": rpc error: code = NotFound desc = could not find container \"66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0\": container with ID starting with 66f460ac6bf4c1376ffc43bb0f85abf86391da6199f48a4743069825c092c6b0 not found: ID does not exist"
Sep 10 21:00:40.897177 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:40.897157    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5dc64c7464-jzhls"]
Sep 10 21:00:41.436589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.436553    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f58bc88fb-4xzlv" podUID="fd89fd26-d84b-4630-a841-fc1262604ba0" containerName="console" containerID="cri-o://28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f" gracePeriod=15
Sep 10 21:00:41.752542 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.752510    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:41.866070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.864829    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9jt7\" (UniqueName: \"kubernetes.io/projected/fd89fd26-d84b-4630-a841-fc1262604ba0-kube-api-access-v9jt7\") pod \"fd89fd26-d84b-4630-a841-fc1262604ba0\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") "
Sep 10 21:00:41.866070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.864909    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-service-ca\") pod \"fd89fd26-d84b-4630-a841-fc1262604ba0\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") "
Sep 10 21:00:41.866070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.864944    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-trusted-ca-bundle\") pod \"fd89fd26-d84b-4630-a841-fc1262604ba0\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") "
Sep 10 21:00:41.866070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.864988    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-oauth-config\") pod \"fd89fd26-d84b-4630-a841-fc1262604ba0\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") "
Sep 10 21:00:41.866070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.865019    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-oauth-serving-cert\") pod \"fd89fd26-d84b-4630-a841-fc1262604ba0\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") "
Sep 10 21:00:41.866070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.865068    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-console-config\") pod \"fd89fd26-d84b-4630-a841-fc1262604ba0\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") "
Sep 10 21:00:41.866070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.865104    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-serving-cert\") pod \"fd89fd26-d84b-4630-a841-fc1262604ba0\" (UID: \"fd89fd26-d84b-4630-a841-fc1262604ba0\") "
Sep 10 21:00:41.867384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.867017    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd89fd26-d84b-4630-a841-fc1262604ba0" (UID: "fd89fd26-d84b-4630-a841-fc1262604ba0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:41.867526 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.867450    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd89fd26-d84b-4630-a841-fc1262604ba0" (UID: "fd89fd26-d84b-4630-a841-fc1262604ba0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:41.867697 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.867589    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd89fd26-d84b-4630-a841-fc1262604ba0" (UID: "fd89fd26-d84b-4630-a841-fc1262604ba0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:41.868095 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.867835    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-console-config" (OuterVolumeSpecName: "console-config") pod "fd89fd26-d84b-4630-a841-fc1262604ba0" (UID: "fd89fd26-d84b-4630-a841-fc1262604ba0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:41.870272 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.870246    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd89fd26-d84b-4630-a841-fc1262604ba0" (UID: "fd89fd26-d84b-4630-a841-fc1262604ba0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:41.870538 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.870517    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd89fd26-d84b-4630-a841-fc1262604ba0" (UID: "fd89fd26-d84b-4630-a841-fc1262604ba0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:41.870622 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.870603    2584 generic.go:356] "Generic (PLEG): container finished" podID="fd89fd26-d84b-4630-a841-fc1262604ba0" containerID="28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f" exitCode=0
Sep 10 21:00:41.870701 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.870680    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f58bc88fb-4xzlv" event={"ID":"fd89fd26-d84b-4630-a841-fc1262604ba0","Type":"ContainerDied","Data":"28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f"}
Sep 10 21:00:41.870887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.870760    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f58bc88fb-4xzlv" event={"ID":"fd89fd26-d84b-4630-a841-fc1262604ba0","Type":"ContainerDied","Data":"6899ccdf827804899dccae30010fb1cc9f8878e875a6b3892fc3ed5285992595"}
Sep 10 21:00:41.870887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.870687    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f58bc88fb-4xzlv"
Sep 10 21:00:41.870887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.870791    2584 scope.go:117] "RemoveContainer" containerID="28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f"
Sep 10 21:00:41.871594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.871537    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd89fd26-d84b-4630-a841-fc1262604ba0-kube-api-access-v9jt7" (OuterVolumeSpecName: "kube-api-access-v9jt7") pod "fd89fd26-d84b-4630-a841-fc1262604ba0" (UID: "fd89fd26-d84b-4630-a841-fc1262604ba0"). InnerVolumeSpecName "kube-api-access-v9jt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:41.882137 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.882114    2584 scope.go:117] "RemoveContainer" containerID="28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f"
Sep 10 21:00:41.882404 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:00:41.882383    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f\": container with ID starting with 28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f not found: ID does not exist" containerID="28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f"
Sep 10 21:00:41.882492 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.882433    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f"} err="failed to get container status \"28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f\": rpc error: code = NotFound desc = could not find container \"28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f\": container with ID starting with 28e2d07e410370c41dde0099cd3ebb741c9743f60bc8c9212d8a010bbcd6a59f not found: ID does not exist"
Sep 10 21:00:41.966597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.966521    2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-console-config\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:41.966597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.966548    2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-serving-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:41.966597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.966558    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9jt7\" (UniqueName: \"kubernetes.io/projected/fd89fd26-d84b-4630-a841-fc1262604ba0-kube-api-access-v9jt7\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:41.966597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.966567    2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:41.966597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.966575    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-trusted-ca-bundle\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:41.966597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.966583    2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd89fd26-d84b-4630-a841-fc1262604ba0-console-oauth-config\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:41.966597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:41.966591    2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd89fd26-d84b-4630-a841-fc1262604ba0-oauth-serving-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:00:42.202944 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:42.202911    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f58bc88fb-4xzlv"]
Sep 10 21:00:42.206158 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:42.206130    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f58bc88fb-4xzlv"]
Sep 10 21:00:42.316664 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:42.316581    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c5445ae-52bb-4a45-a3e8-80b742b26b21" path="/var/lib/kubelet/pods/5c5445ae-52bb-4a45-a3e8-80b742b26b21/volumes"
Sep 10 21:00:42.317058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:42.317037    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd89fd26-d84b-4630-a841-fc1262604ba0" path="/var/lib/kubelet/pods/fd89fd26-d84b-4630-a841-fc1262604ba0/volumes"
Sep 10 21:00:47.284688 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:47.284659    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:47.288913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:47.288889    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-d446cbf8b-wfblv"
Sep 10 21:00:48.776768 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:48.776722    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:48.918529 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:00:48.918494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:09.161584 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:09.161545    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:09.176936 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:09.176908    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:09.983692 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:09.983665    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:18.635889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:18.635810    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:18.734369 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:18.734329    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:27.355876 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:27.355834    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:01:27.356344 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:27.356144    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="prometheus" containerID="cri-o://05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" gracePeriod=600
Sep 10 21:01:27.356344 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:27.356177    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy" containerID="cri-o://5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1" gracePeriod=600
Sep 10 21:01:27.356344 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:27.356239    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213" gracePeriod=600
Sep 10 21:01:27.356344 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:27.356258    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-web" containerID="cri-o://19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855" gracePeriod=600
Sep 10 21:01:27.357016 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:27.356412    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="config-reloader" containerID="cri-o://21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0" gracePeriod=600
Sep 10 21:01:27.357016 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:27.356448    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="thanos-sidecar" containerID="cri-o://12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f" gracePeriod=600
Sep 10 21:01:28.036740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:28.036706    2584 generic.go:356] "Generic (PLEG): container finished" podID="04728475-3abb-43b2-bd79-4b832220aa4c" containerID="19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855" exitCode=0
Sep 10 21:01:28.036740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:28.036733    2584 generic.go:356] "Generic (PLEG): container finished" podID="04728475-3abb-43b2-bd79-4b832220aa4c" containerID="12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f" exitCode=0
Sep 10 21:01:28.036740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:28.036740    2584 generic.go:356] "Generic (PLEG): container finished" podID="04728475-3abb-43b2-bd79-4b832220aa4c" containerID="21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0" exitCode=0
Sep 10 21:01:28.037024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:28.036766    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855"}
Sep 10 21:01:28.037024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:28.036831    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f"}
Sep 10 21:01:28.037024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:28.036857    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0"}
Sep 10 21:01:29.044577 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:29.044542    2584 generic.go:356] "Generic (PLEG): container finished" podID="04728475-3abb-43b2-bd79-4b832220aa4c" containerID="5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213" exitCode=0
Sep 10 21:01:29.044577 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:29.044570    2584 generic.go:356] "Generic (PLEG): container finished" podID="04728475-3abb-43b2-bd79-4b832220aa4c" containerID="5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1" exitCode=0
Sep 10 21:01:29.045011 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:29.044604    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213"}
Sep 10 21:01:29.045011 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:29.044674    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1"}
Sep 10 21:01:29.970677 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:29.970631    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 10 21:01:29.971689 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:29.971654    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 10 21:01:29.972568 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:29.972528    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 10 21:01:29.972667 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:29.972584    2584 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="prometheus" probeResult="unknown"
Sep 10 21:01:30.570580 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:30.570537    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:30.654524 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:30.654487    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:33.716653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:33.716610    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:33.861693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:33.861662    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:34.970830 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:34.970788    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 10 21:01:34.971815 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:34.971773    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 10 21:01:34.972771 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:34.972735    2584 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Sep 10 21:01:34.972845 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:34.972785    2584 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="prometheus" probeResult="unknown"
Sep 10 21:01:36.416006 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.415977    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:36.489085 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489002    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-1\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489085 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489035    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-0\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489085 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489064    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-config-out\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489085 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489080    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-tls-assets\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489152    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-kubelet-serving-ca-bundle\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489194    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-serving-certs-ca-bundle\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489224    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-tls\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489249    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-config\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489278    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489306    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-trusted-ca-bundle\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489336    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-2\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489371    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdq6l\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-kube-api-access-hdq6l\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489405    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-db\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489412    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489462    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-kube-rbac-proxy\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489499    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-web-config\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489527    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-metrics-client-ca\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489555    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489578    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-metrics-client-certs\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489609    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-thanos-prometheus-http-client-file\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.489883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489638    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-grpc-tls\") pod \"04728475-3abb-43b2-bd79-4b832220aa4c\" (UID: \"04728475-3abb-43b2-bd79-4b832220aa4c\") "
Sep 10 21:01:36.490278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.489895    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.490278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.490158    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:01:36.491140 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.491104    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:01:36.491263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.491168    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:01:36.491562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.491528    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:01:36.491975 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.491952    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:01:36.492454 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.492161    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.492454 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.492253    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:01:36.493683 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.493652    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.493795 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.493774    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.493971 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.493948    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:01:36.495535 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.495410    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-config-out" (OuterVolumeSpecName: "config-out") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:01:36.495657 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.495626    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-kube-api-access-hdq6l" (OuterVolumeSpecName: "kube-api-access-hdq6l") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "kube-api-access-hdq6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:01:36.495804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.495776    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.495804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.495795    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:01:36.495932 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.495862    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.496220 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.496191    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-config" (OuterVolumeSpecName: "config") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.496289 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.496243    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.496334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.496288    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.504819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.504798    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-web-config" (OuterVolumeSpecName: "web-config") pod "04728475-3abb-43b2-bd79-4b832220aa4c" (UID: "04728475-3abb-43b2-bd79-4b832220aa4c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:01:36.590698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590649    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590690    2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-config-out\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590701    2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-tls-assets\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590711    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590720    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590729    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590737    2584 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-config\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590745    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590754    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590762    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590770    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdq6l\" (UniqueName: \"kubernetes.io/projected/04728475-3abb-43b2-bd79-4b832220aa4c-kube-api-access-hdq6l\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590779    2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04728475-3abb-43b2-bd79-4b832220aa4c-prometheus-k8s-db\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590787    2584 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-kube-rbac-proxy\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590795    2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-web-config\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590803    2584 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04728475-3abb-43b2-bd79-4b832220aa4c-configmap-metrics-client-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590813    2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590821    2584 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-metrics-client-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590830    2584 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:36.590991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:36.590839    2584 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04728475-3abb-43b2-bd79-4b832220aa4c-secret-grpc-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:01:37.075362 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.075327    2584 generic.go:356] "Generic (PLEG): container finished" podID="04728475-3abb-43b2-bd79-4b832220aa4c" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0" exitCode=0
Sep 10 21:01:37.075558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.075368    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0"}
Sep 10 21:01:37.075558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.075398    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.075558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.075435    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04728475-3abb-43b2-bd79-4b832220aa4c","Type":"ContainerDied","Data":"d6e72a7417869d15b7815e676631a907a7afb35c5d68f1b472616360572e9c21"}
Sep 10 21:01:37.075558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.075475    2584 scope.go:117] "RemoveContainer" containerID="5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213"
Sep 10 21:01:37.085053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.084793    2584 scope.go:117] "RemoveContainer" containerID="5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1"
Sep 10 21:01:37.093072 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.093053    2584 scope.go:117] "RemoveContainer" containerID="19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855"
Sep 10 21:01:37.100132 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.100109    2584 scope.go:117] "RemoveContainer" containerID="12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f"
Sep 10 21:01:37.105957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.105928    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:01:37.107978 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.107891    2584 scope.go:117] "RemoveContainer" containerID="21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0"
Sep 10 21:01:37.108879 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.108857    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:01:37.114491 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.114474    2584 scope.go:117] "RemoveContainer" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0"
Sep 10 21:01:37.121244 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.121227    2584 scope.go:117] "RemoveContainer" containerID="3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8"
Sep 10 21:01:37.127388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.127367    2584 scope.go:117] "RemoveContainer" containerID="5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213"
Sep 10 21:01:37.127718 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:37.127693    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213\": container with ID starting with 5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213 not found: ID does not exist" containerID="5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213"
Sep 10 21:01:37.127784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.127729    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213"} err="failed to get container status \"5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213\": rpc error: code = NotFound desc = could not find container \"5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213\": container with ID starting with 5c92685f141edffcbd1e699cb46f3293dfb7cf38919997f1886e69e8582fd213 not found: ID does not exist"
Sep 10 21:01:37.127784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.127747    2584 scope.go:117] "RemoveContainer" containerID="5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1"
Sep 10 21:01:37.128020 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:37.128000    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1\": container with ID starting with 5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1 not found: ID does not exist" containerID="5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1"
Sep 10 21:01:37.128058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128026    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1"} err="failed to get container status \"5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1\": rpc error: code = NotFound desc = could not find container \"5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1\": container with ID starting with 5f1c3bd585d671c87771286e397c62746bbbbd9e1af3f22cceba8685c0b080a1 not found: ID does not exist"
Sep 10 21:01:37.128058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128043    2584 scope.go:117] "RemoveContainer" containerID="19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855"
Sep 10 21:01:37.128299 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:37.128281    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855\": container with ID starting with 19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855 not found: ID does not exist" containerID="19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855"
Sep 10 21:01:37.128342 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128304    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855"} err="failed to get container status \"19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855\": rpc error: code = NotFound desc = could not find container \"19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855\": container with ID starting with 19de4075aa4347d80910d86065cabb79f45dac8704663b136a2313e8c537c855 not found: ID does not exist"
Sep 10 21:01:37.128342 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128319    2584 scope.go:117] "RemoveContainer" containerID="12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f"
Sep 10 21:01:37.128581 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:37.128564    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f\": container with ID starting with 12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f not found: ID does not exist" containerID="12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f"
Sep 10 21:01:37.128638 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128586    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f"} err="failed to get container status \"12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f\": rpc error: code = NotFound desc = could not find container \"12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f\": container with ID starting with 12ecd6915bc25943aee332951c851a65a3db473cfcdba714275a50ad89b7dc2f not found: ID does not exist"
Sep 10 21:01:37.128638 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128600    2584 scope.go:117] "RemoveContainer" containerID="21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0"
Sep 10 21:01:37.128816 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:37.128798    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0\": container with ID starting with 21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0 not found: ID does not exist" containerID="21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0"
Sep 10 21:01:37.128860 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128826    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0"} err="failed to get container status \"21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0\": rpc error: code = NotFound desc = could not find container \"21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0\": container with ID starting with 21f03fec766f34b12af8cfd7caab67286ec812e43835912997894f0befe237e0 not found: ID does not exist"
Sep 10 21:01:37.128860 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.128839    2584 scope.go:117] "RemoveContainer" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0"
Sep 10 21:01:37.129036 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:37.129020    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0\": container with ID starting with 05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0 not found: ID does not exist" containerID="05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0"
Sep 10 21:01:37.129079 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.129040    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0"} err="failed to get container status \"05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0\": rpc error: code = NotFound desc = could not find container \"05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0\": container with ID starting with 05ec8b5a27a0fdf8cb6e74e55708b3e0b689e107d1d5438dbf6cda8a91a1a5b0 not found: ID does not exist"
Sep 10 21:01:37.129079 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.129053    2584 scope.go:117] "RemoveContainer" containerID="3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8"
Sep 10 21:01:37.129273 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:01:37.129248    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8\": container with ID starting with 3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8 not found: ID does not exist" containerID="3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8"
Sep 10 21:01:37.129352 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.129275    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8"} err="failed to get container status \"3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8\": rpc error: code = NotFound desc = could not find container \"3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8\": container with ID starting with 3c5de9edd5ca5208a65fea357ec1ea9fbae868a512208e91f124adcdf44586e8 not found: ID does not exist"
Sep 10 21:01:37.162176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162142    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:01:37.162412 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162397    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="prometheus"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162429    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="prometheus"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162441    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-web"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162451    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-web"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162461    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c5445ae-52bb-4a45-a3e8-80b742b26b21" containerName="registry"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162471    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c5445ae-52bb-4a45-a3e8-80b742b26b21" containerName="registry"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162482    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="thanos-sidecar"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162490    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="thanos-sidecar"
Sep 10 21:01:37.162510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162506    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="config-reloader"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162515    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="config-reloader"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162524    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-thanos"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162533    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-thanos"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162546    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd89fd26-d84b-4630-a841-fc1262604ba0" containerName="console"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162555    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd89fd26-d84b-4630-a841-fc1262604ba0" containerName="console"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162564    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="init-config-reloader"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162572    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="init-config-reloader"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162581    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162589    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162659    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="prometheus"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162669    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="thanos-sidecar"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162679    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162687    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd89fd26-d84b-4630-a841-fc1262604ba0" containerName="console"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162696    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-web"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162704    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="config-reloader"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162712    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c5445ae-52bb-4a45-a3e8-80b742b26b21" containerName="registry"
Sep 10 21:01:37.162889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.162720    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" containerName="kube-rbac-proxy-thanos"
Sep 10 21:01:37.167310 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.167291    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.171211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.171180    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 21:01:37.171211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.171180    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 21:01:37.171356 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.171280    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-t5frq\""
Sep 10 21:01:37.171498 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.171481    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 21:01:37.173513 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.172465    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-10c0tp8k75j3r\""
Sep 10 21:01:37.173513 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.172746    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 21:01:37.174093 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.173709    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 21:01:37.175668 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.175039    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 21:01:37.176247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.176230    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 21:01:37.178047 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.178026    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 21:01:37.178212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.178030    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 21:01:37.178212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.178115    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 21:01:37.178212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.178036    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 21:01:37.178357 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.178287    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 21:01:37.179753 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.179733    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 21:01:37.185777 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.185752    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 21:01:37.194319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194297    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194430 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194327    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194430 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194345    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194430 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194359    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194430 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194385    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194441    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194480    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnr8h\" (UniqueName: \"kubernetes.io/projected/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-kube-api-access-cnr8h\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194513    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194540    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194565    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194585    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194605    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194666    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194716    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194732    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-config\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194768    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194843    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.194888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.194861    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.211404 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.211376    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:01:37.295140 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295107    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295140 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295139    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295174    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295209    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295257    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295280    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnr8h\" (UniqueName: \"kubernetes.io/projected/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-kube-api-access-cnr8h\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295440    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295496    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295529    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295565    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295594    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295629    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295669    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-config\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295708    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295746    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295758    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295798    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.295957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.295825    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.296311 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.296083    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.296587 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.296459    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.296587 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.296576    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.297463 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.296988    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.297463 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.297105    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.297845 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.297817    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.298370 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.298341    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.299270 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.299243    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.299769 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.299746    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.300509 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.299971    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.300509 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.299977    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.300509 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.300448    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.300509 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.300469    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-config\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.300835 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.300813    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.301242 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.301213    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.301320 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.301272    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.301507 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.301484    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.301717 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.301698    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.304622 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.304602    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnr8h\" (UniqueName: \"kubernetes.io/projected/3a077b0b-ba04-4521-af6c-f9cc8a9936f1-kube-api-access-cnr8h\") pod \"prometheus-k8s-0\" (UID: \"3a077b0b-ba04-4521-af6c-f9cc8a9936f1\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.479450 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.479402    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:37.769308 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:37.769109    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 21:01:37.771882 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:01:37.771854    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a077b0b_ba04_4521_af6c_f9cc8a9936f1.slice/crio-11aa410188c7829a701568e45d9210aa365d554431073ea3d5001c7110515553 WatchSource:0}: Error finding container 11aa410188c7829a701568e45d9210aa365d554431073ea3d5001c7110515553: Status 404 returned error can't find the container with id 11aa410188c7829a701568e45d9210aa365d554431073ea3d5001c7110515553
Sep 10 21:01:38.080539 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:38.080453    2584 generic.go:356] "Generic (PLEG): container finished" podID="3a077b0b-ba04-4521-af6c-f9cc8a9936f1" containerID="0b2eee2b36fbe3236c88ee8ec7c11a530b9ecbde7d99b0cb257e6a45b01839d3" exitCode=0
Sep 10 21:01:38.080704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:38.080541    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerDied","Data":"0b2eee2b36fbe3236c88ee8ec7c11a530b9ecbde7d99b0cb257e6a45b01839d3"}
Sep 10 21:01:38.080704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:38.080618    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerStarted","Data":"11aa410188c7829a701568e45d9210aa365d554431073ea3d5001c7110515553"}
Sep 10 21:01:38.302844 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:38.302809    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04728475-3abb-43b2-bd79-4b832220aa4c" path="/var/lib/kubelet/pods/04728475-3abb-43b2-bd79-4b832220aa4c/volumes"
Sep 10 21:01:39.088704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:39.088664    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerStarted","Data":"fb18d0a266b08f86572d1ed7bd51d461e327a142368c5c55b3a53bfee14ba0f7"}
Sep 10 21:01:39.089110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:39.088758    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerStarted","Data":"cb32b5f506331715299397b8e8bf64a6541137274de9425f6ba538e5a22324e2"}
Sep 10 21:01:39.089110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:39.088785    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerStarted","Data":"a619fc292eba28aca21dbf29ce0c73d78891670cf5f2587cf156e93531ccd92e"}
Sep 10 21:01:39.089110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:39.088823    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerStarted","Data":"4ac35ed4e103f4f90bd320bee66d650d6baa6cda1a8f6746245c410aa359782f"}
Sep 10 21:01:39.089110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:39.088854    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerStarted","Data":"42a663337f1e7c2886eff9afa9d289717c1c676c1ce67ef1cdecb2d5a80a507b"}
Sep 10 21:01:39.089110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:39.088881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a077b0b-ba04-4521-af6c-f9cc8a9936f1","Type":"ContainerStarted","Data":"afc16eb76ff00c9203bb643f928437438489191184450bf1b6fd31202e9534dc"}
Sep 10 21:01:39.131819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:39.131767    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.111856089 podStartE2EDuration="2.131746596s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.890507ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:01:39.13001261 +0000 UTC m=+193.455773553" watchObservedRunningTime="2026-09-10 21:01:39.131746596 +0000 UTC m=+193.457507516"
Sep 10 21:01:42.480413 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:42.480365    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:01:59.016496 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:59.016442    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:59.112005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:01:59.111964    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:05.242979 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:05.242945    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:05.360192 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:05.360160    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:10.134582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:10.134529    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:10.233383 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:10.233340    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:10.362055 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:10.361996    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:18.455714 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:18.455674    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:31.816972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:31.816938    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:32.875321 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:32.875280    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:33.276359 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:33.276325    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:35.813513 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:35.813474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:35.914436 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:35.914389    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:37.043194 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:37.043164    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:37.143000 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:37.142969    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:37.479864 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:37.479825    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:02:37.494770 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:37.494736    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:02:38.053660 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:38.053624    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:38.158345 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:38.158304    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:38.301986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:38.301959    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 21:02:39.860155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:39.860119    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:40.466721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:40.466687    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:42.615498 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:42.615466    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:42.703025 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:42.702995    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:46.019526 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:46.019484    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:46.133239 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:46.133207    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:49.267900 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:49.267815    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:49.645457 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:49.645412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:49.709120 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:49.709088    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:50.404780 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:50.404742    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:54.225220 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:54.225181    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:54.325257 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:54.325220    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:59.520018 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:59.519973    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:59.625474 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:02:59.625412    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:01.425623 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:01.425584    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:17.269636 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:17.269595    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:17.361555 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:17.361520    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:17.821270 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:17.821238    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.069555 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.069514    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.175931 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.175900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.229925 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.229893    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"]
Sep 10 21:03:18.232934 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.232918    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.235998 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.235975    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 21:03:18.236103 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.236070    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 21:03:18.236279 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.236265    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ldf8n\""
Sep 10 21:03:18.236387 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.236370    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 21:03:18.236725 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.236709    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 21:03:18.250600 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.250578    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"]
Sep 10 21:03:18.359311 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.359274    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.362346 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.362325    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-tmp\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.362478 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.362363    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-webhook-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.362478 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.362396    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.362478 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.362458    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwml4\" (UniqueName: \"kubernetes.io/projected/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-kube-api-access-rwml4\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.362624 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.362528    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-manifests\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.463967 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.463874    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-manifests\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.463967 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.463913    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-tmp\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.463967 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.463940    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-webhook-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.464236 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.463970    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.464236 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.464014    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwml4\" (UniqueName: \"kubernetes.io/projected/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-kube-api-access-rwml4\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.464381 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.464355    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-manifests\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.465089 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.465054    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.466354 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.466334    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-tmp\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.466643 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.466622    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-webhook-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.466722 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.466680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.473407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.473382    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwml4\" (UniqueName: \"kubernetes.io/projected/9c037f75-ddf6-4e20-8a0a-eb3f64b26375-kube-api-access-rwml4\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-zlg2c\" (UID: \"9c037f75-ddf6-4e20-8a0a-eb3f64b26375\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.544445 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.544405    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:18.585977 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.585942    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.685803 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.685771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.791073 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.790959    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:18.831120 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.830367    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"]
Sep 10 21:03:18.835094 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:03:18.835066    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c037f75_ddf6_4e20_8a0a_eb3f64b26375.slice/crio-fda4ea3e5cbff0e77ebed95a5aede25bde5484647b897de75f52eb8b76729e73 WatchSource:0}: Error finding container fda4ea3e5cbff0e77ebed95a5aede25bde5484647b897de75f52eb8b76729e73: Status 404 returned error can't find the container with id fda4ea3e5cbff0e77ebed95a5aede25bde5484647b897de75f52eb8b76729e73
Sep 10 21:03:18.890457 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:18.890413    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:19.427001 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:19.426959    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c" event={"ID":"9c037f75-ddf6-4e20-8a0a-eb3f64b26375","Type":"ContainerStarted","Data":"fda4ea3e5cbff0e77ebed95a5aede25bde5484647b897de75f52eb8b76729e73"}
Sep 10 21:03:22.439594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:22.439558    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c" event={"ID":"9c037f75-ddf6-4e20-8a0a-eb3f64b26375","Type":"ContainerStarted","Data":"163fa1ae53446293f5a7b22c39832d51c75cc3c9b76853f50f1778f6fcb8794c"}
Sep 10 21:03:23.444749 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:23.444708    2584 generic.go:356] "Generic (PLEG): container finished" podID="9c037f75-ddf6-4e20-8a0a-eb3f64b26375" containerID="163fa1ae53446293f5a7b22c39832d51c75cc3c9b76853f50f1778f6fcb8794c" exitCode=0
Sep 10 21:03:23.445223 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:23.444797    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c" event={"ID":"9c037f75-ddf6-4e20-8a0a-eb3f64b26375","Type":"ContainerDied","Data":"163fa1ae53446293f5a7b22c39832d51c75cc3c9b76853f50f1778f6fcb8794c"}
Sep 10 21:03:24.449974 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:24.449938    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c" event={"ID":"9c037f75-ddf6-4e20-8a0a-eb3f64b26375","Type":"ContainerStarted","Data":"7ad0de44e91e16f2dea0a981ad0b566a85884e5cceeed92e23bac6037496efed"}
Sep 10 21:03:24.450369 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:24.450033    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:24.473926 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:24.473872    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c" podStartSLOduration=2.050043712 podStartE2EDuration="6.473856711s" totalImagesPullingTime="3.358080418s" totalInitContainerRuntime="1.065732581s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:24.47199023 +0000 UTC m=+298.797751151" watchObservedRunningTime="2026-09-10 21:03:24.473856711 +0000 UTC m=+298.799617631"
Sep 10 21:03:26.094466 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:26.094413    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:03:26.099091 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:26.099057    2584 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 21:03:26.099251 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:26.099151    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:03:30.002788 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:30.002748    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:30.106001 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:30.105960    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:33.417266 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:33.417224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:33.538885 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:33.538850    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:35.455704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:35.455668    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-zlg2c"
Sep 10 21:03:37.914323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:37.914283    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:38.138149 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.138115    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:38.488869 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.488828    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"]
Sep 10 21:03:38.494720 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.494687    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:38.496404 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.496381    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.499405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.499379    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 21:03:38.499780 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.499760    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 21:03:38.518100 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.518071    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"]
Sep 10 21:03:38.630997 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.630961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xlck\" (UniqueName: \"kubernetes.io/projected/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-kube-api-access-6xlck\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.630997 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.631001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tmp\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.631207 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.631097    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.731752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.731715    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.731909 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.731770    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xlck\" (UniqueName: \"kubernetes.io/projected/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-kube-api-access-6xlck\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.731909 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.731795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tmp\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.731909 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:03:38.731868    2584 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 21:03:38.732067 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:03:38.731936    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tls-certs podName:8dc8b3fe-548b-46f1-a775-f1f7c765ac77 nodeName:}" failed. No retries permitted until 2026-09-10 21:03:39.231920567 +0000 UTC m=+313.557681466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tls-certs") pod "kube-auth-proxy-cd8d54847-5d6sk" (UID: "8dc8b3fe-548b-46f1-a775-f1f7c765ac77") : secret "kube-auth-proxy-tls" not found
Sep 10 21:03:38.733980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.733958    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tmp\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.743274 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.743215    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xlck\" (UniqueName: \"kubernetes.io/projected/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-kube-api-access-6xlck\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:38.996322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:38.996224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:39.235704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.235657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:39.238042 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.238019    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dc8b3fe-548b-46f1-a775-f1f7c765ac77-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5d6sk\" (UID: \"8dc8b3fe-548b-46f1-a775-f1f7c765ac77\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:39.406126 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.406091    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"
Sep 10 21:03:39.685235 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.685153    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk"]
Sep 10 21:03:39.691870 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:03:39.691839    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc8b3fe_548b_46f1_a775_f1f7c765ac77.slice/crio-23b1561585b9be0a06e82aa1e5069a0dfcc3e2e09d6b80b7c213f0db07cd46fa WatchSource:0}: Error finding container 23b1561585b9be0a06e82aa1e5069a0dfcc3e2e09d6b80b7c213f0db07cd46fa: Status 404 returned error can't find the container with id 23b1561585b9be0a06e82aa1e5069a0dfcc3e2e09d6b80b7c213f0db07cd46fa
Sep 10 21:03:39.693606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.693591    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:03:39.979997 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.979919    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"]
Sep 10 21:03:39.982809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.982793    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:39.988614 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.988590    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 21:03:39.988759 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.988686    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 21:03:39.988759 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.988686    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 21:03:39.988881 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.988843    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vc6c8\""
Sep 10 21:03:39.989070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:39.989054    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:03:40.003003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.002979    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"]
Sep 10 21:03:40.040984 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.040948    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ebc878-bd86-43f8-a798-d68a5bffc0bd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.040984 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.040986    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.041187 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.041007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jlwm\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-kube-api-access-4jlwm\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.041187 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.041065    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.041187 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.041090    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.041187 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.041111    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.041187 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.041149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.142218 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.142180    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ebc878-bd86-43f8-a798-d68a5bffc0bd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.142405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.142242    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.142405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.142271    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jlwm\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-kube-api-access-4jlwm\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.142405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.142326    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.142595 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.142476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.142595 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.142506    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.142595 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.142536    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.143138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.143114    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.144930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.144904    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.145061 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.145044    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ebc878-bd86-43f8-a798-d68a5bffc0bd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.145157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.145137    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.145200 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.145186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.167663 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.167635    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.176488 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.176467    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jlwm\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-kube-api-access-4jlwm\") pod \"istiod-openshift-gateway-7b7995d5cd-l65sg\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.291707 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.291612    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:40.468694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.468549    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:40.509681 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.509639    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk" event={"ID":"8dc8b3fe-548b-46f1-a775-f1f7c765ac77","Type":"ContainerStarted","Data":"23b1561585b9be0a06e82aa1e5069a0dfcc3e2e09d6b80b7c213f0db07cd46fa"}
Sep 10 21:03:40.575557 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:40.575457    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"]
Sep 10 21:03:40.579280 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:03:40.579241    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ebc878_bd86_43f8_a798_d68a5bffc0bd.slice/crio-2598620b8188f0ef19aaafd7a1371d9050f32f3c2dff9bfe45d272deabd8bbe9 WatchSource:0}: Error finding container 2598620b8188f0ef19aaafd7a1371d9050f32f3c2dff9bfe45d272deabd8bbe9: Status 404 returned error can't find the container with id 2598620b8188f0ef19aaafd7a1371d9050f32f3c2dff9bfe45d272deabd8bbe9
Sep 10 21:03:41.113408 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:41.113375    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:41.519102 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:41.519058    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg" event={"ID":"29ebc878-bd86-43f8-a798-d68a5bffc0bd","Type":"ContainerStarted","Data":"2598620b8188f0ef19aaafd7a1371d9050f32f3c2dff9bfe45d272deabd8bbe9"}
Sep 10 21:03:42.525923 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:42.525875    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk" event={"ID":"8dc8b3fe-548b-46f1-a775-f1f7c765ac77","Type":"ContainerStarted","Data":"f9059b34a539c818ae02c54d65626be44685d807f342a4b35d13be8ceaee60f6"}
Sep 10 21:03:42.544302 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:42.544169    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5d6sk" podStartSLOduration=1.94496929 podStartE2EDuration="4.544147976s" totalImagesPullingTime="2.599178686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:42.54272209 +0000 UTC m=+316.868483015" watchObservedRunningTime="2026-09-10 21:03:42.544147976 +0000 UTC m=+316.869908898"
Sep 10 21:03:44.040190 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.040147    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:03:44.040515 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.040217    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:03:44.517828 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.517784    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:44.542169 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.542132    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg" event={"ID":"29ebc878-bd86-43f8-a798-d68a5bffc0bd","Type":"ContainerStarted","Data":"da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec"}
Sep 10 21:03:44.542362 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.542253    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:44.564763 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.564705    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg" podStartSLOduration=2.107046041 podStartE2EDuration="5.564688956s" totalImagesPullingTime="3.457642915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:44.562921514 +0000 UTC m=+318.888682437" watchObservedRunningTime="2026-09-10 21:03:44.564688956 +0000 UTC m=+318.890449877"
Sep 10 21:03:44.618395 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.618355    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:44.827673 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.827598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:44.967291 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:44.967253    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:45.467848 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:45.467815    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:45.548101 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:45.548070    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:03:48.886320 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:48.886274    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:48.984741 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:48.984696    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:49.082577 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:49.082539    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:49.436537 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:49.436504    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:49.910440 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:49.910383    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:59.145971 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.145935    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:59.267545 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.267511    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"]
Sep 10 21:03:59.269645 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.269628    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.271843 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.271816    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vhc5p\""
Sep 10 21:03:59.272222 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.272201    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 21:03:59.274875 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.274856    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"]
Sep 10 21:03:59.277301 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.277283    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:03:59.281777 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.281761    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 21:03:59.282223 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.282207    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tzglv\""
Sep 10 21:03:59.288953 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.288930    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"]
Sep 10 21:03:59.291441 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.291394    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:59.292724 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.292700    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"]
Sep 10 21:03:59.408723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.408621    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf7ml\" (UniqueName: \"kubernetes.io/projected/13c47c34-0c80-4fae-9200-98fae6ef10ca-kube-api-access-sf7ml\") pod \"kserve-controller-manager-d6f4b65b8-gzcv5\" (UID: \"13c47c34-0c80-4fae-9200-98fae6ef10ca\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.408723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.408669    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a14f04c0-88a8-447a-bddd-c536626ea4f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5tb8w\" (UID: \"a14f04c0-88a8-447a-bddd-c536626ea4f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:03:59.408957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.408766    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13c47c34-0c80-4fae-9200-98fae6ef10ca-cert\") pod \"kserve-controller-manager-d6f4b65b8-gzcv5\" (UID: \"13c47c34-0c80-4fae-9200-98fae6ef10ca\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.408957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.408865    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjcc4\" (UniqueName: \"kubernetes.io/projected/a14f04c0-88a8-447a-bddd-c536626ea4f9-kube-api-access-xjcc4\") pod \"llmisvc-controller-manager-777c6bd7d5-5tb8w\" (UID: \"a14f04c0-88a8-447a-bddd-c536626ea4f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:03:59.409791 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.409766    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:59.510260 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.510224    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjcc4\" (UniqueName: \"kubernetes.io/projected/a14f04c0-88a8-447a-bddd-c536626ea4f9-kube-api-access-xjcc4\") pod \"llmisvc-controller-manager-777c6bd7d5-5tb8w\" (UID: \"a14f04c0-88a8-447a-bddd-c536626ea4f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:03:59.510470 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.510282    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf7ml\" (UniqueName: \"kubernetes.io/projected/13c47c34-0c80-4fae-9200-98fae6ef10ca-kube-api-access-sf7ml\") pod \"kserve-controller-manager-d6f4b65b8-gzcv5\" (UID: \"13c47c34-0c80-4fae-9200-98fae6ef10ca\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.510470 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.510302    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a14f04c0-88a8-447a-bddd-c536626ea4f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5tb8w\" (UID: \"a14f04c0-88a8-447a-bddd-c536626ea4f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:03:59.510470 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.510326    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13c47c34-0c80-4fae-9200-98fae6ef10ca-cert\") pod \"kserve-controller-manager-d6f4b65b8-gzcv5\" (UID: \"13c47c34-0c80-4fae-9200-98fae6ef10ca\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.510470 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:03:59.510440    2584 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 21:03:59.510661 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:03:59.510515    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a14f04c0-88a8-447a-bddd-c536626ea4f9-cert podName:a14f04c0-88a8-447a-bddd-c536626ea4f9 nodeName:}" failed. No retries permitted until 2026-09-10 21:04:00.010500348 +0000 UTC m=+334.336261253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a14f04c0-88a8-447a-bddd-c536626ea4f9-cert") pod "llmisvc-controller-manager-777c6bd7d5-5tb8w" (UID: "a14f04c0-88a8-447a-bddd-c536626ea4f9") : secret "llmisvc-webhook-server-cert" not found
Sep 10 21:03:59.512756 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.512733    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13c47c34-0c80-4fae-9200-98fae6ef10ca-cert\") pod \"kserve-controller-manager-d6f4b65b8-gzcv5\" (UID: \"13c47c34-0c80-4fae-9200-98fae6ef10ca\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.520075 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.520054    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjcc4\" (UniqueName: \"kubernetes.io/projected/a14f04c0-88a8-447a-bddd-c536626ea4f9-kube-api-access-xjcc4\") pod \"llmisvc-controller-manager-777c6bd7d5-5tb8w\" (UID: \"a14f04c0-88a8-447a-bddd-c536626ea4f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:03:59.520217 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.520199    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf7ml\" (UniqueName: \"kubernetes.io/projected/13c47c34-0c80-4fae-9200-98fae6ef10ca-kube-api-access-sf7ml\") pod \"kserve-controller-manager-d6f4b65b8-gzcv5\" (UID: \"13c47c34-0c80-4fae-9200-98fae6ef10ca\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.579853 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.579830    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:03:59.912941 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.912904    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:59.920545 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:03:59.920520    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"]
Sep 10 21:03:59.922278 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:03:59.922253    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c47c34_0c80_4fae_9200_98fae6ef10ca.slice/crio-78d665182c17482a6e6256c29f68ac1992443f470d4091cb64438f7eb6cfe474 WatchSource:0}: Error finding container 78d665182c17482a6e6256c29f68ac1992443f470d4091cb64438f7eb6cfe474: Status 404 returned error can't find the container with id 78d665182c17482a6e6256c29f68ac1992443f470d4091cb64438f7eb6cfe474
Sep 10 21:04:00.014847 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.014820    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a14f04c0-88a8-447a-bddd-c536626ea4f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5tb8w\" (UID: \"a14f04c0-88a8-447a-bddd-c536626ea4f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:04:00.017247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.017222    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a14f04c0-88a8-447a-bddd-c536626ea4f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5tb8w\" (UID: \"a14f04c0-88a8-447a-bddd-c536626ea4f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:04:00.187620 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.187534    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:04:00.462872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.462844    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"]
Sep 10 21:04:00.465511 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:04:00.465481    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda14f04c0_88a8_447a_bddd_c536626ea4f9.slice/crio-7abff9d0b68c03ce6a4952cb11d4250c1ebb13b2f8ac8d565f20acdbd9da864e WatchSource:0}: Error finding container 7abff9d0b68c03ce6a4952cb11d4250c1ebb13b2f8ac8d565f20acdbd9da864e: Status 404 returned error can't find the container with id 7abff9d0b68c03ce6a4952cb11d4250c1ebb13b2f8ac8d565f20acdbd9da864e
Sep 10 21:04:00.501687 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.501647    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:00.523964 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.523936    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"]
Sep 10 21:04:00.524165 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.524137    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg" podUID="29ebc878-bd86-43f8-a798-d68a5bffc0bd" containerName="discovery" containerID="cri-o://da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec" gracePeriod=30
Sep 10 21:04:00.547586 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.547557    2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-l65sg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.18:8080/ready\": dial tcp 10.132.0.18:8080: connect: connection refused" start-of-body=
Sep 10 21:04:00.547711 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.547615    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg" podUID="29ebc878-bd86-43f8-a798-d68a5bffc0bd" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.18:8080/ready\": dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 10 21:04:00.600672 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.600630    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w" event={"ID":"a14f04c0-88a8-447a-bddd-c536626ea4f9","Type":"ContainerStarted","Data":"7abff9d0b68c03ce6a4952cb11d4250c1ebb13b2f8ac8d565f20acdbd9da864e"}
Sep 10 21:04:00.601994 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.601957    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5" event={"ID":"13c47c34-0c80-4fae-9200-98fae6ef10ca","Type":"ContainerStarted","Data":"78d665182c17482a6e6256c29f68ac1992443f470d4091cb64438f7eb6cfe474"}
Sep 10 21:04:00.630137 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.630093    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:00.774410 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.774387    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:04:00.821505 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.821471    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-kubeconfig\") pod \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") "
Sep 10 21:04:00.821665 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.821517    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-dns-cert\") pod \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") "
Sep 10 21:04:00.821665 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.821549    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-token\") pod \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") "
Sep 10 21:04:00.821665 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.821608    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ebc878-bd86-43f8-a798-d68a5bffc0bd-local-certs\") pod \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") "
Sep 10 21:04:00.821665 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.821633    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-ca-configmap\") pod \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") "
Sep 10 21:04:00.821878 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.821704    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jlwm\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-kube-api-access-4jlwm\") pod \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") "
Sep 10 21:04:00.821878 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.821730    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-cacerts\") pod \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\" (UID: \"29ebc878-bd86-43f8-a798-d68a5bffc0bd\") "
Sep 10 21:04:00.822374 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.822339    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "29ebc878-bd86-43f8-a798-d68a5bffc0bd" (UID: "29ebc878-bd86-43f8-a798-d68a5bffc0bd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:04:00.825348 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.825313    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ebc878-bd86-43f8-a798-d68a5bffc0bd-local-certs" (OuterVolumeSpecName: "local-certs") pod "29ebc878-bd86-43f8-a798-d68a5bffc0bd" (UID: "29ebc878-bd86-43f8-a798-d68a5bffc0bd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:00.825527 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.825507    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "29ebc878-bd86-43f8-a798-d68a5bffc0bd" (UID: "29ebc878-bd86-43f8-a798-d68a5bffc0bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:04:00.825598 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.825513    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "29ebc878-bd86-43f8-a798-d68a5bffc0bd" (UID: "29ebc878-bd86-43f8-a798-d68a5bffc0bd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:04:00.825882 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.825853    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-cacerts" (OuterVolumeSpecName: "cacerts") pod "29ebc878-bd86-43f8-a798-d68a5bffc0bd" (UID: "29ebc878-bd86-43f8-a798-d68a5bffc0bd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:04:00.825972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.825945    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-kube-api-access-4jlwm" (OuterVolumeSpecName: "kube-api-access-4jlwm") pod "29ebc878-bd86-43f8-a798-d68a5bffc0bd" (UID: "29ebc878-bd86-43f8-a798-d68a5bffc0bd"). InnerVolumeSpecName "kube-api-access-4jlwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:04:00.826325 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.826299    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "29ebc878-bd86-43f8-a798-d68a5bffc0bd" (UID: "29ebc878-bd86-43f8-a798-d68a5bffc0bd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:04:00.922521 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.922483    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jlwm\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-kube-api-access-4jlwm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:04:00.922521 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.922516    2584 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-cacerts\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:04:00.922521 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.922527    2584 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-kubeconfig\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:04:00.922798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.922536    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-dns-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:04:00.922798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.922547    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-token\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:04:00.922798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.922555    2584 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ebc878-bd86-43f8-a798-d68a5bffc0bd-local-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:04:00.922798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:00.922563    2584 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/29ebc878-bd86-43f8-a798-d68a5bffc0bd-istio-csr-ca-configmap\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:04:01.609505 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:01.609466    2584 generic.go:356] "Generic (PLEG): container finished" podID="29ebc878-bd86-43f8-a798-d68a5bffc0bd" containerID="da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec" exitCode=0
Sep 10 21:04:01.609946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:01.609532    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"
Sep 10 21:04:01.609946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:01.609550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg" event={"ID":"29ebc878-bd86-43f8-a798-d68a5bffc0bd","Type":"ContainerDied","Data":"da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec"}
Sep 10 21:04:01.609946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:01.609605    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg" event={"ID":"29ebc878-bd86-43f8-a798-d68a5bffc0bd","Type":"ContainerDied","Data":"2598620b8188f0ef19aaafd7a1371d9050f32f3c2dff9bfe45d272deabd8bbe9"}
Sep 10 21:04:01.609946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:01.609632    2584 scope.go:117] "RemoveContainer" containerID="da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec"
Sep 10 21:04:01.638927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:01.638891    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"]
Sep 10 21:04:01.644767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:01.644731    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l65sg"]
Sep 10 21:04:02.080798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:02.080672    2584 scope.go:117] "RemoveContainer" containerID="da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec"
Sep 10 21:04:02.081152 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:04:02.081124    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec\": container with ID starting with da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec not found: ID does not exist" containerID="da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec"
Sep 10 21:04:02.081247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:02.081165    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec"} err="failed to get container status \"da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec\": rpc error: code = NotFound desc = could not find container \"da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec\": container with ID starting with da801a1d7848279558c083560952ca97e11e9c37a6dbf5994d000db27463e0ec not found: ID does not exist"
Sep 10 21:04:02.300789 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:02.300738    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ebc878-bd86-43f8-a798-d68a5bffc0bd" path="/var/lib/kubelet/pods/29ebc878-bd86-43f8-a798-d68a5bffc0bd/volumes"
Sep 10 21:04:02.617136 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:02.617091    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5" event={"ID":"13c47c34-0c80-4fae-9200-98fae6ef10ca","Type":"ContainerStarted","Data":"f1677685660b22581e0c56ee478e0a049d46f600c9ddc7bac7df3c542c3531a8"}
Sep 10 21:04:02.617614 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:02.617375    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:04:02.640201 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:02.640146    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5" podStartSLOduration=1.336277307 podStartE2EDuration="3.640130219s" totalImagesPullingTime="2.303852912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:04:02.638053626 +0000 UTC m=+336.963814547" watchObservedRunningTime="2026-09-10 21:04:02.640130219 +0000 UTC m=+336.965891140"
Sep 10 21:04:03.622084 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:03.621992    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w" event={"ID":"a14f04c0-88a8-447a-bddd-c536626ea4f9","Type":"ContainerStarted","Data":"9287da4c76470c4b1b16fc07c42435f8b8bc11d034aba14efc649bd508a7b368"}
Sep 10 21:04:03.622453 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:03.622075    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:04:03.638482 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:03.638388    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w" podStartSLOduration=2.176254367 podStartE2EDuration="4.638367445s" totalImagesPullingTime="2.462113078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:04:03.638249353 +0000 UTC m=+337.964010272" watchObservedRunningTime="2026-09-10 21:04:03.638367445 +0000 UTC m=+337.964128370"
Sep 10 21:04:33.627255 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:33.627223    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gzcv5"
Sep 10 21:04:34.631302 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:34.631269    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tb8w"
Sep 10 21:04:50.026515 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:50.026474    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:50.113335 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:04:50.113301    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:05.644148 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.644117    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:05.839573 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.839538    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7564bb99bd-g8tcb"]
Sep 10 21:05:05.839832 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.839819    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ebc878-bd86-43f8-a798-d68a5bffc0bd" containerName="discovery"
Sep 10 21:05:05.839878 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.839833    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ebc878-bd86-43f8-a798-d68a5bffc0bd" containerName="discovery"
Sep 10 21:05:05.839912 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.839881    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ebc878-bd86-43f8-a798-d68a5bffc0bd" containerName="discovery"
Sep 10 21:05:05.841844 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.841820    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.844642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.844623    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 21:05:05.844967 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.844950    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 21:05:05.846045 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.846021    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 21:05:05.846215 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.846189    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 21:05:05.846886 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.846872    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 21:05:05.846951 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.846896    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 21:05:05.847100 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.847082    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dtlm8\""
Sep 10 21:05:05.848465 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.848444    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 21:05:05.852458 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.852441    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 21:05:05.878131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.878102    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7564bb99bd-g8tcb"]
Sep 10 21:05:05.937536 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.937506    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88596fd8-abda-45f8-8457-dac6cafbf095-console-serving-cert\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.937684 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.937561    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-trusted-ca-bundle\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.937684 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.937622    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-console-config\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.937797 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.937683    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-service-ca\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.937797 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.937706    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-oauth-serving-cert\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.937797 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.937745    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88596fd8-abda-45f8-8457-dac6cafbf095-console-oauth-config\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.937894 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.937811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drm9t\" (UniqueName: \"kubernetes.io/projected/88596fd8-abda-45f8-8457-dac6cafbf095-kube-api-access-drm9t\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:05.996585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:05.996554    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:06.038673 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.038647    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88596fd8-abda-45f8-8457-dac6cafbf095-console-serving-cert\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.038796 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.038695    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-trusted-ca-bundle\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.038796 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.038733    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-console-config\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.038796 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.038775    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-service-ca\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.038943 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.038796    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-oauth-serving-cert\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.038943 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.038833    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88596fd8-abda-45f8-8457-dac6cafbf095-console-oauth-config\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.038943 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.038862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drm9t\" (UniqueName: \"kubernetes.io/projected/88596fd8-abda-45f8-8457-dac6cafbf095-kube-api-access-drm9t\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.039620 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.039594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-oauth-serving-cert\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.039733 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.039594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-service-ca\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.039733 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.039679    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-console-config\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.039853 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.039741    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88596fd8-abda-45f8-8457-dac6cafbf095-trusted-ca-bundle\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.041245 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.041215    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88596fd8-abda-45f8-8457-dac6cafbf095-console-serving-cert\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.041357 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.041321    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88596fd8-abda-45f8-8457-dac6cafbf095-console-oauth-config\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.052183 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.052161    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drm9t\" (UniqueName: \"kubernetes.io/projected/88596fd8-abda-45f8-8457-dac6cafbf095-kube-api-access-drm9t\") pod \"console-7564bb99bd-g8tcb\" (UID: \"88596fd8-abda-45f8-8457-dac6cafbf095\") " pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.152192 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.152148    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:06.440250 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.440214    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7564bb99bd-g8tcb"]
Sep 10 21:05:06.444257 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:06.444227    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88596fd8_abda_45f8_8457_dac6cafbf095.slice/crio-686f53138d0de12763b098316e1fe3cf3672cbd837c05ef795a352271f8e9cec WatchSource:0}: Error finding container 686f53138d0de12763b098316e1fe3cf3672cbd837c05ef795a352271f8e9cec: Status 404 returned error can't find the container with id 686f53138d0de12763b098316e1fe3cf3672cbd837c05ef795a352271f8e9cec
Sep 10 21:05:06.637216 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.637183    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:06.856002 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.855914    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7564bb99bd-g8tcb" event={"ID":"88596fd8-abda-45f8-8457-dac6cafbf095","Type":"ContainerStarted","Data":"fdf77856392352c066f2eac7cce0fae1baccce64fea7113b97c95b9fec6ed531"}
Sep 10 21:05:06.856002 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.855977    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7564bb99bd-g8tcb" event={"ID":"88596fd8-abda-45f8-8457-dac6cafbf095","Type":"ContainerStarted","Data":"686f53138d0de12763b098316e1fe3cf3672cbd837c05ef795a352271f8e9cec"}
Sep 10 21:05:06.885822 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:06.885767    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7564bb99bd-g8tcb" podStartSLOduration=1.885748873 podStartE2EDuration="1.885748873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:06.885022473 +0000 UTC m=+401.210783395" watchObservedRunningTime="2026-09-10 21:05:06.885748873 +0000 UTC m=+401.211509793"
Sep 10 21:05:07.621264 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:07.621215    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:12.380956 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.380916    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"]
Sep 10 21:05:12.383183 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.383160    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.399501 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.399479    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:05:12.399501 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.399481    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:05:12.399679 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.399563    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jjpzr\""
Sep 10 21:05:12.426245 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.426216    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"]
Sep 10 21:05:12.494597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.494550    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.494768 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.494649    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn2wx\" (UniqueName: \"kubernetes.io/projected/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-kube-api-access-bn2wx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.595229 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.595191    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.595380 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.595251    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2wx\" (UniqueName: \"kubernetes.io/projected/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-kube-api-access-bn2wx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.595603 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.595584    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.613483 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.613453    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn2wx\" (UniqueName: \"kubernetes.io/projected/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-kube-api-access-bn2wx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.693564 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.693533    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:12.981622 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:12.981549    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"]
Sep 10 21:05:12.988869 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:12.988840    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd7dfe0_c5c0_428c_8c20_f1d220c129bf.slice/crio-d670ce5a29e5415ba697a03cc2cb0a9b69be979a4f89fb0375478b9fa7c5b82f WatchSource:0}: Error finding container d670ce5a29e5415ba697a03cc2cb0a9b69be979a4f89fb0375478b9fa7c5b82f: Status 404 returned error can't find the container with id d670ce5a29e5415ba697a03cc2cb0a9b69be979a4f89fb0375478b9fa7c5b82f
Sep 10 21:05:13.199762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:13.199719    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:13.886296 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:13.886255    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq" event={"ID":"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf","Type":"ContainerStarted","Data":"d670ce5a29e5415ba697a03cc2cb0a9b69be979a4f89fb0375478b9fa7c5b82f"}
Sep 10 21:05:15.253722 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.253686    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"]
Sep 10 21:05:15.256165 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.256141    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"
Sep 10 21:05:15.262749 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.262683    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-x2gmj\""
Sep 10 21:05:15.262749 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.262737    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 21:05:15.287226 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.287200    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"]
Sep 10 21:05:15.318235 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.318206    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6hll\" (UniqueName: \"kubernetes.io/projected/5b3552ad-b1c4-4e48-a97e-862e12149dba-kube-api-access-m6hll\") pod \"dns-operator-controller-manager-7b84cd7c9c-mkq46\" (UID: \"5b3552ad-b1c4-4e48-a97e-862e12149dba\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"
Sep 10 21:05:15.419097 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.419007    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6hll\" (UniqueName: \"kubernetes.io/projected/5b3552ad-b1c4-4e48-a97e-862e12149dba-kube-api-access-m6hll\") pod \"dns-operator-controller-manager-7b84cd7c9c-mkq46\" (UID: \"5b3552ad-b1c4-4e48-a97e-862e12149dba\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"
Sep 10 21:05:15.447961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.447929    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6hll\" (UniqueName: \"kubernetes.io/projected/5b3552ad-b1c4-4e48-a97e-862e12149dba-kube-api-access-m6hll\") pod \"dns-operator-controller-manager-7b84cd7c9c-mkq46\" (UID: \"5b3552ad-b1c4-4e48-a97e-862e12149dba\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"
Sep 10 21:05:15.568023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.567893    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"
Sep 10 21:05:15.749638 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:15.749597    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:16.153097 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.153062    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:16.153271 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.153147    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:16.158367 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.158342    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:16.268186 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.268148    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:16.902244 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.902206    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq" event={"ID":"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf","Type":"ContainerStarted","Data":"c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8"}
Sep 10 21:05:16.902406 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.902299    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:16.906080 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.906057    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7564bb99bd-g8tcb"
Sep 10 21:05:16.934282 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:16.934223    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq" podStartSLOduration=1.116437469 podStartE2EDuration="4.934208914s" totalImagesPullingTime="3.817771445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:16.932749119 +0000 UTC m=+411.258510063" watchObservedRunningTime="2026-09-10 21:05:16.934208914 +0000 UTC m=+411.259969834"
Sep 10 21:05:17.039735 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:17.039655    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"]
Sep 10 21:05:17.043174 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:17.043146    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3552ad_b1c4_4e48_a97e_862e12149dba.slice/crio-e8e71e62a37f05ac88cd6174f86c4c9409cfcc9fb3dc96d89105aa5509451b87 WatchSource:0}: Error finding container e8e71e62a37f05ac88cd6174f86c4c9409cfcc9fb3dc96d89105aa5509451b87: Status 404 returned error can't find the container with id e8e71e62a37f05ac88cd6174f86c4c9409cfcc9fb3dc96d89105aa5509451b87
Sep 10 21:05:17.278262 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:17.278224    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:17.462092 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:17.462059    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:17.628551 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:17.628513    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:17.908789 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:17.908744    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46" event={"ID":"5b3552ad-b1c4-4e48-a97e-862e12149dba","Type":"ContainerStarted","Data":"e8e71e62a37f05ac88cd6174f86c4c9409cfcc9fb3dc96d89105aa5509451b87"}
Sep 10 21:05:18.914739 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:18.914699    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46" event={"ID":"5b3552ad-b1c4-4e48-a97e-862e12149dba","Type":"ContainerStarted","Data":"39e5581f332f0650e5e34b4f19f5b8753907b77b873bb667ffac6f5e0497f0e0"}
Sep 10 21:05:18.915159 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:18.914784    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"
Sep 10 21:05:18.941800 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:18.941743    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46" podStartSLOduration=2.487349841 podStartE2EDuration="3.941724163s" totalImagesPullingTime="1.454374322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:18.941348056 +0000 UTC m=+413.267108990" watchObservedRunningTime="2026-09-10 21:05:18.941724163 +0000 UTC m=+413.267485085"
Sep 10 21:05:19.063804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.063771    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-w27hq"]
Sep 10 21:05:19.065854 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.065838    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-w27hq"
Sep 10 21:05:19.083644 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.083615    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-w27hq"]
Sep 10 21:05:19.099965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.099942    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4dg6v\""
Sep 10 21:05:19.152817 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.152789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcnr\" (UniqueName: \"kubernetes.io/projected/f02d78cf-792b-466e-bba9-9406d2257d58-kube-api-access-bpcnr\") pod \"authorino-operator-76b478bd78-w27hq\" (UID: \"f02d78cf-792b-466e-bba9-9406d2257d58\") " pod="kuadrant-system/authorino-operator-76b478bd78-w27hq"
Sep 10 21:05:19.254200 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.254121    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcnr\" (UniqueName: \"kubernetes.io/projected/f02d78cf-792b-466e-bba9-9406d2257d58-kube-api-access-bpcnr\") pod \"authorino-operator-76b478bd78-w27hq\" (UID: \"f02d78cf-792b-466e-bba9-9406d2257d58\") " pod="kuadrant-system/authorino-operator-76b478bd78-w27hq"
Sep 10 21:05:19.275961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.275936    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcnr\" (UniqueName: \"kubernetes.io/projected/f02d78cf-792b-466e-bba9-9406d2257d58-kube-api-access-bpcnr\") pod \"authorino-operator-76b478bd78-w27hq\" (UID: \"f02d78cf-792b-466e-bba9-9406d2257d58\") " pod="kuadrant-system/authorino-operator-76b478bd78-w27hq"
Sep 10 21:05:19.375728 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.375697    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-w27hq"
Sep 10 21:05:19.648222 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.648189    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-w27hq"]
Sep 10 21:05:19.651255 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:19.651228    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02d78cf_792b_466e_bba9_9406d2257d58.slice/crio-241bbc01a67b4e8cb6eca7ff7e585e6e9048194bbcd4dd904e3466f96730dc27 WatchSource:0}: Error finding container 241bbc01a67b4e8cb6eca7ff7e585e6e9048194bbcd4dd904e3466f96730dc27: Status 404 returned error can't find the container with id 241bbc01a67b4e8cb6eca7ff7e585e6e9048194bbcd4dd904e3466f96730dc27
Sep 10 21:05:19.809526 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.809488    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:19.814490 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.814457    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"]
Sep 10 21:05:19.817034 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.817018    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:19.819829 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.819810    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 21:05:19.819945 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.819849    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 21:05:19.819945 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.819847    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pmnkr\""
Sep 10 21:05:19.828956 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.828934    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"]
Sep 10 21:05:19.920484 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.920374    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-w27hq" event={"ID":"f02d78cf-792b-466e-bba9-9406d2257d58","Type":"ContainerStarted","Data":"241bbc01a67b4e8cb6eca7ff7e585e6e9048194bbcd4dd904e3466f96730dc27"}
Sep 10 21:05:19.960830 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.960797    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:19.960972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.960844    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:19.960972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:19.960922    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbm4h\" (UniqueName: \"kubernetes.io/projected/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-kube-api-access-lbm4h\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.021161 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.021119    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:20.061305 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.061280    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.061453 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.061338    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.061453 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.061380    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbm4h\" (UniqueName: \"kubernetes.io/projected/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-kube-api-access-lbm4h\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.062066 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.062046    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.063810 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.063787    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.073115 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.073088    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbm4h\" (UniqueName: \"kubernetes.io/projected/6faf7168-06ee-44fc-bc37-4c7aae6e8c5b-kube-api-access-lbm4h\") pod \"kuadrant-console-plugin-fbdd4f445-zs2hn\" (UID: \"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.127005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.126980    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"
Sep 10 21:05:20.207034 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.207000    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:20.418691 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.415522    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn"]
Sep 10 21:05:20.928138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.927868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn" event={"ID":"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b","Type":"ContainerStarted","Data":"66b511b91a3ef50c3232c0971374e0c02d8ed1c8a7ba88e86e8d2de7805b51eb"}
Sep 10 21:05:20.930610 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.930513    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-w27hq" event={"ID":"f02d78cf-792b-466e-bba9-9406d2257d58","Type":"ContainerStarted","Data":"4178945061a75944331fbcbcd90c593380f65a6969a614283eb90d8c0da702cf"}
Sep 10 21:05:20.930726 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.930636    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-w27hq"
Sep 10 21:05:20.971828 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:20.971637    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-w27hq" podStartSLOduration=0.761825205 podStartE2EDuration="1.971617136s" totalImagesPullingTime="1.209791931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:20.969708071 +0000 UTC m=+415.295468994" watchObservedRunningTime="2026-09-10 21:05:20.971617136 +0000 UTC m=+415.297378058"
Sep 10 21:05:22.078710 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.078679    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"]
Sep 10 21:05:22.081636 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.081613    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:22.084380 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.084356    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kbxnb\""
Sep 10 21:05:22.098114 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.098085    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"]
Sep 10 21:05:22.181180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.181143    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpw6p\" (UniqueName: \"kubernetes.io/projected/8f358237-ba78-44fc-b908-ee1ab1dd705a-kube-api-access-fpw6p\") pod \"limitador-operator-controller-manager-74495c69f8-8xl9x\" (UID: \"8f358237-ba78-44fc-b908-ee1ab1dd705a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:22.282494 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.282458    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpw6p\" (UniqueName: \"kubernetes.io/projected/8f358237-ba78-44fc-b908-ee1ab1dd705a-kube-api-access-fpw6p\") pod \"limitador-operator-controller-manager-74495c69f8-8xl9x\" (UID: \"8f358237-ba78-44fc-b908-ee1ab1dd705a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:22.304626 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.304599    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpw6p\" (UniqueName: \"kubernetes.io/projected/8f358237-ba78-44fc-b908-ee1ab1dd705a-kube-api-access-fpw6p\") pod \"limitador-operator-controller-manager-74495c69f8-8xl9x\" (UID: \"8f358237-ba78-44fc-b908-ee1ab1dd705a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:22.394294 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:22.394213    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:23.277485 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:23.277441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:23.878893 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:23.878856    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:24.004906 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:24.004853    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:24.103451 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:24.103397    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:24.850638 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:24.850614    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"]
Sep 10 21:05:24.851057 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:24.850891    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:24.853056 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:24.853025    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f358237_ba78_44fc_b908_ee1ab1dd705a.slice/crio-f517e50d6f348b624ce1f8f4d85f6338741a3df74eb77c9dda30de0c7d73145d WatchSource:0}: Error finding container f517e50d6f348b624ce1f8f4d85f6338741a3df74eb77c9dda30de0c7d73145d: Status 404 returned error can't find the container with id f517e50d6f348b624ce1f8f4d85f6338741a3df74eb77c9dda30de0c7d73145d
Sep 10 21:05:24.954245 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:24.954207    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn" event={"ID":"6faf7168-06ee-44fc-bc37-4c7aae6e8c5b","Type":"ContainerStarted","Data":"ea39572030573ac48b1a1a0a079959a57eb294d12ea30397c91cfe7049be15f5"}
Sep 10 21:05:24.955486 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:24.955461    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x" event={"ID":"8f358237-ba78-44fc-b908-ee1ab1dd705a","Type":"ContainerStarted","Data":"f517e50d6f348b624ce1f8f4d85f6338741a3df74eb77c9dda30de0c7d73145d"}
Sep 10 21:05:24.981128 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:24.981083    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zs2hn" podStartSLOduration=1.798215346 podStartE2EDuration="5.981069224s" totalImagesPullingTime="4.182853878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:24.980373805 +0000 UTC m=+419.306134726" watchObservedRunningTime="2026-09-10 21:05:24.981069224 +0000 UTC m=+419.306830145"
Sep 10 21:05:25.545930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:25.545888    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:25.681926 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:25.681890    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:25.926697 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:25.926661    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:26.051240 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:26.051203    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:26.208784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:26.208701    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:26.432153 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:26.432117    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:26.595275 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:26.595181    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:26.967392 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:26.967358    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x" event={"ID":"8f358237-ba78-44fc-b908-ee1ab1dd705a","Type":"ContainerStarted","Data":"b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982"}
Sep 10 21:05:26.967834 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:26.967500    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:26.993676 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:26.993627    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x" podStartSLOduration=3.738181559 podStartE2EDuration="4.993612994s" totalImagesPullingTime="1.255431435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:26.993462326 +0000 UTC m=+421.319223249" watchObservedRunningTime="2026-09-10 21:05:26.993612994 +0000 UTC m=+421.319373916"
Sep 10 21:05:27.911582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:27.911551    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:29.922675 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:29.922639    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mkq46"
Sep 10 21:05:30.410484 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.410445    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"]
Sep 10 21:05:30.410686 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.410658    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" containerName="manager" containerID="cri-o://c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8" gracePeriod=2
Sep 10 21:05:30.412084 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.412058    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"]
Sep 10 21:05:30.416279 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.416242    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.454612 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.454575    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"]
Sep 10 21:05:30.455092 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.455069    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" containerName="manager"
Sep 10 21:05:30.455180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.455094    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" containerName="manager"
Sep 10 21:05:30.455233 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.455199    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" containerName="manager"
Sep 10 21:05:30.458328 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.458309    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.464858 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.464834    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"]
Sep 10 21:05:30.465042 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.465013    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" containerName="manager" containerID="cri-o://b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982" gracePeriod=2
Sep 10 21:05:30.469359 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.469326    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"]
Sep 10 21:05:30.485524 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.485491    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"]
Sep 10 21:05:30.509304 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.509262    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.510074 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.510055    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"]
Sep 10 21:05:30.510346 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.510335    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" containerName="manager"
Sep 10 21:05:30.510379 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.510348    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" containerName="manager"
Sep 10 21:05:30.510413 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.510399    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" containerName="manager"
Sep 10 21:05:30.513255 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.513242    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"
Sep 10 21:05:30.516578 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.516548    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:30.519630 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.519602    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.525944 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.525921    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.530878 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.530855    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:30.546943 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.546908    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"]
Sep 10 21:05:30.551618 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.551594    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54dw5\" (UniqueName: \"kubernetes.io/projected/ee235831-aa7e-40e3-997b-294a293f91e1-kube-api-access-54dw5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-dv6mm\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.551767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.551745    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee235831-aa7e-40e3-997b-294a293f91e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-dv6mm\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.653101 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.653028    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vjr8\" (UniqueName: \"kubernetes.io/projected/e1479352-9c28-4bfd-92ae-412a7fb51bf8-kube-api-access-6vjr8\") pod \"limitador-operator-controller-manager-74495c69f8-8ttsw\" (UID: \"e1479352-9c28-4bfd-92ae-412a7fb51bf8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"
Sep 10 21:05:30.653261 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.653167    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee235831-aa7e-40e3-997b-294a293f91e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-dv6mm\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.653261 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.653211    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54dw5\" (UniqueName: \"kubernetes.io/projected/ee235831-aa7e-40e3-997b-294a293f91e1-kube-api-access-54dw5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-dv6mm\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.653840 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.653818    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee235831-aa7e-40e3-997b-294a293f91e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-dv6mm\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.664045 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.663974    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54dw5\" (UniqueName: \"kubernetes.io/projected/ee235831-aa7e-40e3-997b-294a293f91e1-kube-api-access-54dw5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-dv6mm\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.704938 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.704916    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.707291 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.707271    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:30.709634 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.709613    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.715430 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.715385    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:30.717908 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.717887    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.720734 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.720707    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:30.754152 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.754122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vjr8\" (UniqueName: \"kubernetes.io/projected/e1479352-9c28-4bfd-92ae-412a7fb51bf8-kube-api-access-6vjr8\") pod \"limitador-operator-controller-manager-74495c69f8-8ttsw\" (UID: \"e1479352-9c28-4bfd-92ae-412a7fb51bf8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"
Sep 10 21:05:30.767624 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.767596    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vjr8\" (UniqueName: \"kubernetes.io/projected/e1479352-9c28-4bfd-92ae-412a7fb51bf8-kube-api-access-6vjr8\") pod \"limitador-operator-controller-manager-74495c69f8-8ttsw\" (UID: \"e1479352-9c28-4bfd-92ae-412a7fb51bf8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"
Sep 10 21:05:30.848519 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.848484    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:30.854559 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.854538    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpw6p\" (UniqueName: \"kubernetes.io/projected/8f358237-ba78-44fc-b908-ee1ab1dd705a-kube-api-access-fpw6p\") pod \"8f358237-ba78-44fc-b908-ee1ab1dd705a\" (UID: \"8f358237-ba78-44fc-b908-ee1ab1dd705a\") "
Sep 10 21:05:30.854654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.854586    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-extensions-socket-volume\") pod \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") "
Sep 10 21:05:30.854704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.854670    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn2wx\" (UniqueName: \"kubernetes.io/projected/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-kube-api-access-bn2wx\") pod \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\" (UID: \"cdd7dfe0-c5c0-428c-8c20-f1d220c129bf\") "
Sep 10 21:05:30.855057 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.855025    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" (UID: "cdd7dfe0-c5c0-428c-8c20-f1d220c129bf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:05:30.856093 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.856073    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"
Sep 10 21:05:30.856770 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.856748    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-kube-api-access-bn2wx" (OuterVolumeSpecName: "kube-api-access-bn2wx") pod "cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" (UID: "cdd7dfe0-c5c0-428c-8c20-f1d220c129bf"). InnerVolumeSpecName "kube-api-access-bn2wx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:05:30.856887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.856863    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f358237-ba78-44fc-b908-ee1ab1dd705a-kube-api-access-fpw6p" (OuterVolumeSpecName: "kube-api-access-fpw6p") pod "8f358237-ba78-44fc-b908-ee1ab1dd705a" (UID: "8f358237-ba78-44fc-b908-ee1ab1dd705a"). InnerVolumeSpecName "kube-api-access-fpw6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:05:30.956121 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.956087    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn2wx\" (UniqueName: \"kubernetes.io/projected/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-kube-api-access-bn2wx\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:05:30.956121 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.956119    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpw6p\" (UniqueName: \"kubernetes.io/projected/8f358237-ba78-44fc-b908-ee1ab1dd705a-kube-api-access-fpw6p\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:05:30.956535 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.956131    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf-extensions-socket-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:05:30.988309 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.988278    2584 generic.go:356] "Generic (PLEG): container finished" podID="8f358237-ba78-44fc-b908-ee1ab1dd705a" containerID="b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982" exitCode=0
Sep 10 21:05:30.988481 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.988326    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:30.988481 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.988369    2584 scope.go:117] "RemoveContainer" containerID="b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982"
Sep 10 21:05:30.990075 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.990054    2584 generic.go:356] "Generic (PLEG): container finished" podID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" containerID="c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8" exitCode=0
Sep 10 21:05:30.990180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.990113    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.991309 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.991282    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.995406 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.995376    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:30.996533 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.996508    2584 scope.go:117] "RemoveContainer" containerID="b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982"
Sep 10 21:05:30.996800 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.996778    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982\": container with ID starting with b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982 not found: ID does not exist" containerID="b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982"
Sep 10 21:05:30.996883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.996812    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982"} err="failed to get container status \"b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982\": rpc error: code = NotFound desc = could not find container \"b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982\": container with ID starting with b97e3686bfa0b533c0ac5e2421aa69c5883b6b85b881bb06c85b9faddab57982 not found: ID does not exist"
Sep 10 21:05:30.996883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:30.996835    2584 scope.go:117] "RemoveContainer" containerID="c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8"
Sep 10 21:05:30.997188 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.997164    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:30.998960 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:30.998932    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:31.003110 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.003082    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:31.004931 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.004913    2584 scope.go:117] "RemoveContainer" containerID="c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8"
Sep 10 21:05:31.004988 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.004968    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:31.005219 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.005201    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8\": container with ID starting with c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8 not found: ID does not exist" containerID="c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8"
Sep 10 21:05:31.005268 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.005227    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8"} err="failed to get container status \"c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8\": rpc error: code = NotFound desc = could not find container \"c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8\": container with ID starting with c6a789272588dbd5e223fd0d0a4a326ece81fb159429d381192731bb963b56e8 not found: ID does not exist"
Sep 10 21:05:31.006826 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.006798    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:31.008489 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.008469    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:31.128341 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.128304    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"]
Sep 10 21:05:31.131509 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:31.131476    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee235831_aa7e_40e3_997b_294a293f91e1.slice/crio-7fb45dd4831a04d742523df9d74bd1b441e60e02037ba29f2a8d10fec9aade21 WatchSource:0}: Error finding container 7fb45dd4831a04d742523df9d74bd1b441e60e02037ba29f2a8d10fec9aade21: Status 404 returned error can't find the container with id 7fb45dd4831a04d742523df9d74bd1b441e60e02037ba29f2a8d10fec9aade21
Sep 10 21:05:31.141090 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.141064    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"]
Sep 10 21:05:31.144846 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:31.144819    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1479352_9c28_4bfd_92ae_412a7fb51bf8.slice/crio-29ba2eb696baa4f2f212ce64e2b79c3e7583ac2bf6050eac2e5c4a8f9cb453ae WatchSource:0}: Error finding container 29ba2eb696baa4f2f212ce64e2b79c3e7583ac2bf6050eac2e5c4a8f9cb453ae: Status 404 returned error can't find the container with id 29ba2eb696baa4f2f212ce64e2b79c3e7583ac2bf6050eac2e5c4a8f9cb453ae
Sep 10 21:05:31.940760 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.940733    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-w27hq"
Sep 10 21:05:31.942998 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.942968    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:31.967493 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.967458    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:31.996944 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.996908    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw" event={"ID":"e1479352-9c28-4bfd-92ae-412a7fb51bf8","Type":"ContainerStarted","Data":"81ac8e55f84ee1b01baab7b6350ab6f16c3e3a7f0d3660716208525d6b2cc436"}
Sep 10 21:05:31.997099 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.996962    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw" event={"ID":"e1479352-9c28-4bfd-92ae-412a7fb51bf8","Type":"ContainerStarted","Data":"29ba2eb696baa4f2f212ce64e2b79c3e7583ac2bf6050eac2e5c4a8f9cb453ae"}
Sep 10 21:05:31.997099 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.997019    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"
Sep 10 21:05:31.998936 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.998904    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm" event={"ID":"ee235831-aa7e-40e3-997b-294a293f91e1","Type":"ContainerStarted","Data":"868b4c5dd67a86752b7c099c369f7b797895ef9fc9f74fc7cc381c36503fa5f4"}
Sep 10 21:05:31.999068 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.998968    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm" event={"ID":"ee235831-aa7e-40e3-997b-294a293f91e1","Type":"ContainerStarted","Data":"7fb45dd4831a04d742523df9d74bd1b441e60e02037ba29f2a8d10fec9aade21"}
Sep 10 21:05:31.999068 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:31.999000    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:31.999173 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:31.999119    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:32.006116 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:32.006082    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:32.058058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:32.057999    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw" podStartSLOduration=2.057977906 podStartE2EDuration="2.057977906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:32.057350753 +0000 UTC m=+426.383111674" watchObservedRunningTime="2026-09-10 21:05:32.057977906 +0000 UTC m=+426.383738827"
Sep 10 21:05:32.097603 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:32.097560    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-89xcq\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-89xcq"
Sep 10 21:05:32.099302 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:05:32.099266    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-8xl9x\" is forbidden: User \"system:node:ip-10-0-128-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-196.ec2.internal' and this object" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8xl9x"
Sep 10 21:05:32.099784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:32.099744    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm" podStartSLOduration=2.099729742 podStartE2EDuration="2.099729742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:32.095766266 +0000 UTC m=+426.421527186" watchObservedRunningTime="2026-09-10 21:05:32.099729742 +0000 UTC m=+426.425490662"
Sep 10 21:05:32.301645 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:32.301566    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f358237-ba78-44fc-b908-ee1ab1dd705a" path="/var/lib/kubelet/pods/8f358237-ba78-44fc-b908-ee1ab1dd705a/volumes"
Sep 10 21:05:32.301939 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:32.301926    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd7dfe0-c5c0-428c-8c20-f1d220c129bf" path="/var/lib/kubelet/pods/cdd7dfe0-c5c0-428c-8c20-f1d220c129bf/volumes"
Sep 10 21:05:33.230550 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:33.230510    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:33.472077 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:33.472044    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:39.052037 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:39.052003    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:39.150535 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:39.150469    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:42.034247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:42.034203    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:42.130584 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:42.130545    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:43.008157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:43.008122    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8ttsw"
Sep 10 21:05:43.008325 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:43.008196    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:46.692977 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.692946    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"]
Sep 10 21:05:46.693383 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.693120    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm" podUID="ee235831-aa7e-40e3-997b-294a293f91e1" containerName="manager" containerID="cri-o://868b4c5dd67a86752b7c099c369f7b797895ef9fc9f74fc7cc381c36503fa5f4" gracePeriod=10
Sep 10 21:05:46.870406 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.870368    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:46.934249 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.934216    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"]
Sep 10 21:05:46.937573 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.937553    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:46.960387 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.960319    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"]
Sep 10 21:05:46.986850 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.986822    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nqzn\" (UniqueName: \"kubernetes.io/projected/d3903999-961e-4545-bd07-c48ceca8c1d4-kube-api-access-9nqzn\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nlsqc\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:46.986982 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:46.986878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3903999-961e-4545-bd07-c48ceca8c1d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nlsqc\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:47.005260 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.005223    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:47.067874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.067841    2584 generic.go:356] "Generic (PLEG): container finished" podID="ee235831-aa7e-40e3-997b-294a293f91e1" containerID="868b4c5dd67a86752b7c099c369f7b797895ef9fc9f74fc7cc381c36503fa5f4" exitCode=0
Sep 10 21:05:47.068023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.067879    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm" event={"ID":"ee235831-aa7e-40e3-997b-294a293f91e1","Type":"ContainerDied","Data":"868b4c5dd67a86752b7c099c369f7b797895ef9fc9f74fc7cc381c36503fa5f4"}
Sep 10 21:05:47.087363 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.087333    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nqzn\" (UniqueName: \"kubernetes.io/projected/d3903999-961e-4545-bd07-c48ceca8c1d4-kube-api-access-9nqzn\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nlsqc\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:47.087482 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.087378    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3903999-961e-4545-bd07-c48ceca8c1d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nlsqc\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:47.087750 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.087733    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3903999-961e-4545-bd07-c48ceca8c1d4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nlsqc\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:47.099728 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.099708    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nqzn\" (UniqueName: \"kubernetes.io/projected/d3903999-961e-4545-bd07-c48ceca8c1d4-kube-api-access-9nqzn\") pod \"kuadrant-operator-controller-manager-5769f99d9f-nlsqc\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:47.109394 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.109369    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:47.290853 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.290776    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:47.485685 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.485645    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:47.489948 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.489924    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54dw5\" (UniqueName: \"kubernetes.io/projected/ee235831-aa7e-40e3-997b-294a293f91e1-kube-api-access-54dw5\") pod \"ee235831-aa7e-40e3-997b-294a293f91e1\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") "
Sep 10 21:05:47.490081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.490018    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee235831-aa7e-40e3-997b-294a293f91e1-extensions-socket-volume\") pod \"ee235831-aa7e-40e3-997b-294a293f91e1\" (UID: \"ee235831-aa7e-40e3-997b-294a293f91e1\") "
Sep 10 21:05:47.490375 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.490351    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee235831-aa7e-40e3-997b-294a293f91e1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ee235831-aa7e-40e3-997b-294a293f91e1" (UID: "ee235831-aa7e-40e3-997b-294a293f91e1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:05:47.491817 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.491790    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee235831-aa7e-40e3-997b-294a293f91e1-kube-api-access-54dw5" (OuterVolumeSpecName: "kube-api-access-54dw5") pod "ee235831-aa7e-40e3-997b-294a293f91e1" (UID: "ee235831-aa7e-40e3-997b-294a293f91e1"). InnerVolumeSpecName "kube-api-access-54dw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:05:47.590618 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.590543    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee235831-aa7e-40e3-997b-294a293f91e1-extensions-socket-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:05:47.590618 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.590572    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54dw5\" (UniqueName: \"kubernetes.io/projected/ee235831-aa7e-40e3-997b-294a293f91e1-kube-api-access-54dw5\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:05:47.599299 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:47.599275    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"]
Sep 10 21:05:47.602049 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:05:47.602020    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3903999_961e_4545_bd07_c48ceca8c1d4.slice/crio-4ce6a4e927c10316fad6514eeb3dfb3eaa0f302c55cdc52c0f4546a117df6616 WatchSource:0}: Error finding container 4ce6a4e927c10316fad6514eeb3dfb3eaa0f302c55cdc52c0f4546a117df6616: Status 404 returned error can't find the container with id 4ce6a4e927c10316fad6514eeb3dfb3eaa0f302c55cdc52c0f4546a117df6616
Sep 10 21:05:48.074922 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.074886    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc" event={"ID":"d3903999-961e-4545-bd07-c48ceca8c1d4","Type":"ContainerStarted","Data":"c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4"}
Sep 10 21:05:48.075378 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.074952    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc" event={"ID":"d3903999-961e-4545-bd07-c48ceca8c1d4","Type":"ContainerStarted","Data":"4ce6a4e927c10316fad6514eeb3dfb3eaa0f302c55cdc52c0f4546a117df6616"}
Sep 10 21:05:48.075378 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.074982    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:48.076601 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.076580    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"
Sep 10 21:05:48.076719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.076576    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm" event={"ID":"ee235831-aa7e-40e3-997b-294a293f91e1","Type":"ContainerDied","Data":"7fb45dd4831a04d742523df9d74bd1b441e60e02037ba29f2a8d10fec9aade21"}
Sep 10 21:05:48.076719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.076717    2584 scope.go:117] "RemoveContainer" containerID="868b4c5dd67a86752b7c099c369f7b797895ef9fc9f74fc7cc381c36503fa5f4"
Sep 10 21:05:48.099242 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.099191    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc" podStartSLOduration=2.099172271 podStartE2EDuration="2.099172271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:48.097799465 +0000 UTC m=+442.423560398" watchObservedRunningTime="2026-09-10 21:05:48.099172271 +0000 UTC m=+442.424933193"
Sep 10 21:05:48.118584 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.118559    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"]
Sep 10 21:05:48.124975 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.124954    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-dv6mm"]
Sep 10 21:05:48.300091 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:48.300052    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee235831-aa7e-40e3-997b-294a293f91e1" path="/var/lib/kubelet/pods/ee235831-aa7e-40e3-997b-294a293f91e1/volumes"
Sep 10 21:05:53.311998 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:53.311966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:53.406006 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:53.405965    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:59.084804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:59.084771    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:05:59.846807 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:59.846763    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:59.945644 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:05:59.945594    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:02.850508 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:02.850471    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:03.003392 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:03.003355    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:03.153656 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:03.153616    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:03.316565 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:03.316530    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:07.852251 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:07.852215    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:07.953368 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:07.953329    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:08.259840 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:08.259798    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:08.360371 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:08.360332    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:08.655333 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:08.655304    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:08.757476 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:08.757440    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:15.107562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:15.107519    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:15.166677 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:15.166640    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:16.738826 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.738792    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:16.750189 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.750154    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tf979"]
Sep 10 21:06:16.750530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.750515    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee235831-aa7e-40e3-997b-294a293f91e1" containerName="manager"
Sep 10 21:06:16.750530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.750530    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee235831-aa7e-40e3-997b-294a293f91e1" containerName="manager"
Sep 10 21:06:16.750627 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.750581    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee235831-aa7e-40e3-997b-294a293f91e1" containerName="manager"
Sep 10 21:06:16.752784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.752765    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:16.754756 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.754735    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 21:06:16.762074 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.762052    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tf979"]
Sep 10 21:06:16.821344 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.821307    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afa56e6c-f2b9-4102-b0ba-4a985f131e44-config-file\") pod \"limitador-limitador-8588749977-tf979\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") " pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:16.821529 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.821370    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lrkz\" (UniqueName: \"kubernetes.io/projected/afa56e6c-f2b9-4102-b0ba-4a985f131e44-kube-api-access-8lrkz\") pod \"limitador-limitador-8588749977-tf979\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") " pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:16.855615 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.855584    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tf979"]
Sep 10 21:06:16.868138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.868103    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:16.922300 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.922265    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afa56e6c-f2b9-4102-b0ba-4a985f131e44-config-file\") pod \"limitador-limitador-8588749977-tf979\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") " pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:16.922493 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.922315    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lrkz\" (UniqueName: \"kubernetes.io/projected/afa56e6c-f2b9-4102-b0ba-4a985f131e44-kube-api-access-8lrkz\") pod \"limitador-limitador-8588749977-tf979\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") " pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:16.922904 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.922883    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afa56e6c-f2b9-4102-b0ba-4a985f131e44-config-file\") pod \"limitador-limitador-8588749977-tf979\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") " pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:16.936275 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.936246    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lrkz\" (UniqueName: \"kubernetes.io/projected/afa56e6c-f2b9-4102-b0ba-4a985f131e44-kube-api-access-8lrkz\") pod \"limitador-limitador-8588749977-tf979\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") " pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:16.964190 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:16.964151    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:17.062694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:17.062609    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:17.338022 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:17.337998    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tf979"]
Sep 10 21:06:17.340657 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:06:17.340624    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa56e6c_f2b9_4102_b0ba_4a985f131e44.slice/crio-1c125b86ca5fd13719acdfe8faf9d0a123bc9200c079830bc37f761134d02ec5 WatchSource:0}: Error finding container 1c125b86ca5fd13719acdfe8faf9d0a123bc9200c079830bc37f761134d02ec5: Status 404 returned error can't find the container with id 1c125b86ca5fd13719acdfe8faf9d0a123bc9200c079830bc37f761134d02ec5
Sep 10 21:06:17.632834 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:17.632733    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:17.742802 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:17.742770    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:18.223906 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:18.223867    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tf979" event={"ID":"afa56e6c-f2b9-4102-b0ba-4a985f131e44","Type":"ContainerStarted","Data":"1c125b86ca5fd13719acdfe8faf9d0a123bc9200c079830bc37f761134d02ec5"}
Sep 10 21:06:19.230572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:19.230530    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tf979" event={"ID":"afa56e6c-f2b9-4102-b0ba-4a985f131e44","Type":"ContainerStarted","Data":"55a69dabe6df2fcc23a7f7bb4fb1b1e0e2997d7ab33d3fd88657ae02caddd05d"}
Sep 10 21:06:19.230969 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:19.230672    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:19.252587 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:19.252536    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-tf979" podStartSLOduration=1.978981703 podStartE2EDuration="3.252519722s" totalImagesPullingTime="1.273538019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:19.251131239 +0000 UTC m=+473.576892162" watchObservedRunningTime="2026-09-10 21:06:19.252519722 +0000 UTC m=+473.578280642"
Sep 10 21:06:21.675817 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:21.675773    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:21.781182 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:21.781146    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:24.656828 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:24.656786    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:24.758706 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:24.758672    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:25.462147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:25.462104    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:30.236229 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:30.236198    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:30.898529 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:30.898496    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tf979"]
Sep 10 21:06:30.898726 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:30.898699    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-tf979" podUID="afa56e6c-f2b9-4102-b0ba-4a985f131e44" containerName="limitador" containerID="cri-o://55a69dabe6df2fcc23a7f7bb4fb1b1e0e2997d7ab33d3fd88657ae02caddd05d" gracePeriod=30
Sep 10 21:06:31.283668 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:31.283628    2584 generic.go:356] "Generic (PLEG): container finished" podID="afa56e6c-f2b9-4102-b0ba-4a985f131e44" containerID="55a69dabe6df2fcc23a7f7bb4fb1b1e0e2997d7ab33d3fd88657ae02caddd05d" exitCode=0
Sep 10 21:06:31.283668 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:31.283676    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tf979" event={"ID":"afa56e6c-f2b9-4102-b0ba-4a985f131e44","Type":"ContainerDied","Data":"55a69dabe6df2fcc23a7f7bb4fb1b1e0e2997d7ab33d3fd88657ae02caddd05d"}
Sep 10 21:06:31.838732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:31.838702    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:31.955991 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:31.955958    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afa56e6c-f2b9-4102-b0ba-4a985f131e44-config-file\") pod \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") "
Sep 10 21:06:31.956172 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:31.956032    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lrkz\" (UniqueName: \"kubernetes.io/projected/afa56e6c-f2b9-4102-b0ba-4a985f131e44-kube-api-access-8lrkz\") pod \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\" (UID: \"afa56e6c-f2b9-4102-b0ba-4a985f131e44\") "
Sep 10 21:06:31.956367 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:31.956343    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afa56e6c-f2b9-4102-b0ba-4a985f131e44-config-file" (OuterVolumeSpecName: "config-file") pod "afa56e6c-f2b9-4102-b0ba-4a985f131e44" (UID: "afa56e6c-f2b9-4102-b0ba-4a985f131e44"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:06:31.958018 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:31.957995    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afa56e6c-f2b9-4102-b0ba-4a985f131e44-kube-api-access-8lrkz" (OuterVolumeSpecName: "kube-api-access-8lrkz") pod "afa56e6c-f2b9-4102-b0ba-4a985f131e44" (UID: "afa56e6c-f2b9-4102-b0ba-4a985f131e44"). InnerVolumeSpecName "kube-api-access-8lrkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:06:32.057347 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:32.057303    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lrkz\" (UniqueName: \"kubernetes.io/projected/afa56e6c-f2b9-4102-b0ba-4a985f131e44-kube-api-access-8lrkz\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:06:32.057347 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:32.057342    2584 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afa56e6c-f2b9-4102-b0ba-4a985f131e44-config-file\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:06:32.289531 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:32.289446    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-tf979"
Sep 10 21:06:32.289971 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:32.289445    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-tf979" event={"ID":"afa56e6c-f2b9-4102-b0ba-4a985f131e44","Type":"ContainerDied","Data":"1c125b86ca5fd13719acdfe8faf9d0a123bc9200c079830bc37f761134d02ec5"}
Sep 10 21:06:32.289971 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:32.289580    2584 scope.go:117] "RemoveContainer" containerID="55a69dabe6df2fcc23a7f7bb4fb1b1e0e2997d7ab33d3fd88657ae02caddd05d"
Sep 10 21:06:32.313988 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:32.313959    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tf979"]
Sep 10 21:06:32.321503 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:32.321477    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-tf979"]
Sep 10 21:06:34.301328 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:34.301294    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afa56e6c-f2b9-4102-b0ba-4a985f131e44" path="/var/lib/kubelet/pods/afa56e6c-f2b9-4102-b0ba-4a985f131e44/volumes"
Sep 10 21:06:36.403311 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.403277    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:36.500125 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.500088    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:36.609348 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.609314    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:36.706642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.706556    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:36.834224 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.834184    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"]
Sep 10 21:06:36.834590 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.834572    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afa56e6c-f2b9-4102-b0ba-4a985f131e44" containerName="limitador"
Sep 10 21:06:36.834677 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.834591    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="afa56e6c-f2b9-4102-b0ba-4a985f131e44" containerName="limitador"
Sep 10 21:06:36.834737 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.834685    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="afa56e6c-f2b9-4102-b0ba-4a985f131e44" containerName="limitador"
Sep 10 21:06:36.837354 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.837334    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:36.839514 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.839412    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:06:36.840322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.840160    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:06:36.840322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.840194    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-z6xzv\""
Sep 10 21:06:36.840322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.840206    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 21:06:36.847382 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.847358    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"]
Sep 10 21:06:36.880505 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:36.880465    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:37.003070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.002990    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksvql\" (UniqueName: \"kubernetes.io/projected/21e1e403-ce0b-4005-9b1b-6534321d5c81-kube-api-access-ksvql\") pod \"postgres-69fb4594cf-zbftp\" (UID: \"21e1e403-ce0b-4005-9b1b-6534321d5c81\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:37.003070 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.003033    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/21e1e403-ce0b-4005-9b1b-6534321d5c81-data\") pod \"postgres-69fb4594cf-zbftp\" (UID: \"21e1e403-ce0b-4005-9b1b-6534321d5c81\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:37.103901 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.103866    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksvql\" (UniqueName: \"kubernetes.io/projected/21e1e403-ce0b-4005-9b1b-6534321d5c81-kube-api-access-ksvql\") pod \"postgres-69fb4594cf-zbftp\" (UID: \"21e1e403-ce0b-4005-9b1b-6534321d5c81\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:37.104075 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.103911    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/21e1e403-ce0b-4005-9b1b-6534321d5c81-data\") pod \"postgres-69fb4594cf-zbftp\" (UID: \"21e1e403-ce0b-4005-9b1b-6534321d5c81\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:37.104281 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.104262    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/21e1e403-ce0b-4005-9b1b-6534321d5c81-data\") pod \"postgres-69fb4594cf-zbftp\" (UID: \"21e1e403-ce0b-4005-9b1b-6534321d5c81\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:37.112427 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.112406    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksvql\" (UniqueName: \"kubernetes.io/projected/21e1e403-ce0b-4005-9b1b-6534321d5c81-kube-api-access-ksvql\") pod \"postgres-69fb4594cf-zbftp\" (UID: \"21e1e403-ce0b-4005-9b1b-6534321d5c81\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:37.149234 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.149207    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:37.426791 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:37.426764    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"]
Sep 10 21:06:37.427965 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:06:37.427939    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e1e403_ce0b_4005_9b1b_6534321d5c81.slice/crio-6403d9ada2fa687b48bf9927e8dce5ab68bfd8e25cde2418392c46c7fa403da2 WatchSource:0}: Error finding container 6403d9ada2fa687b48bf9927e8dce5ab68bfd8e25cde2418392c46c7fa403da2: Status 404 returned error can't find the container with id 6403d9ada2fa687b48bf9927e8dce5ab68bfd8e25cde2418392c46c7fa403da2
Sep 10 21:06:38.318795 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:38.318748    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp" event={"ID":"21e1e403-ce0b-4005-9b1b-6534321d5c81","Type":"ContainerStarted","Data":"6403d9ada2fa687b48bf9927e8dce5ab68bfd8e25cde2418392c46c7fa403da2"}
Sep 10 21:06:40.328842 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:40.328806    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp" event={"ID":"21e1e403-ce0b-4005-9b1b-6534321d5c81","Type":"ContainerStarted","Data":"613604f675d65bf938634c57fae9524f278723fa445f234f65420c9419f413db"}
Sep 10 21:06:40.329220 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:40.328913    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:40.345531 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:40.345478    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp" podStartSLOduration=2.304527801 podStartE2EDuration="4.345462002s" totalImagesPullingTime="2.040934201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:40.344069895 +0000 UTC m=+494.669830819" watchObservedRunningTime="2026-09-10 21:06:40.345462002 +0000 UTC m=+494.671222917"
Sep 10 21:06:46.361727 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.361698    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-zbftp"
Sep 10 21:06:46.656530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.656487    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:46.868247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.868215    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-wmrt8"]
Sep 10 21:06:46.878095 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.878066    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wmrt8"
Sep 10 21:06:46.878963 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.878939    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-wmrt8"]
Sep 10 21:06:46.881657 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.881633    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6hnqb\""
Sep 10 21:06:46.889520 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.889494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57n62\" (UniqueName: \"kubernetes.io/projected/45db3a49-e902-4eeb-b33b-825d23c7597a-kube-api-access-57n62\") pod \"authorino-557f876f69-wmrt8\" (UID: \"45db3a49-e902-4eeb-b33b-825d23c7597a\") " pod="kuadrant-system/authorino-557f876f69-wmrt8"
Sep 10 21:06:46.990737 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.990640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57n62\" (UniqueName: \"kubernetes.io/projected/45db3a49-e902-4eeb-b33b-825d23c7597a-kube-api-access-57n62\") pod \"authorino-557f876f69-wmrt8\" (UID: \"45db3a49-e902-4eeb-b33b-825d23c7597a\") " pod="kuadrant-system/authorino-557f876f69-wmrt8"
Sep 10 21:06:46.999806 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:46.999777    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57n62\" (UniqueName: \"kubernetes.io/projected/45db3a49-e902-4eeb-b33b-825d23c7597a-kube-api-access-57n62\") pod \"authorino-557f876f69-wmrt8\" (UID: \"45db3a49-e902-4eeb-b33b-825d23c7597a\") " pod="kuadrant-system/authorino-557f876f69-wmrt8"
Sep 10 21:06:47.098899 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.098861    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wmrt8"]
Sep 10 21:06:47.099135 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.099121    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wmrt8"
Sep 10 21:06:47.204351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.204295    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:47.288981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.288885    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-744c9c7c76-xpk5f"]
Sep 10 21:06:47.293516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.293490    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.295666 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.295643    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 21:06:47.301915 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.301894    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-744c9c7c76-xpk5f"]
Sep 10 21:06:47.369205 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.369180    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wmrt8"]
Sep 10 21:06:47.372414 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:06:47.372382    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45db3a49_e902_4eeb_b33b_825d23c7597a.slice/crio-8b56f47494c3e9fb19512073b0344a51631fd77897ddd51851c0f00f792ea1ce WatchSource:0}: Error finding container 8b56f47494c3e9fb19512073b0344a51631fd77897ddd51851c0f00f792ea1ce: Status 404 returned error can't find the container with id 8b56f47494c3e9fb19512073b0344a51631fd77897ddd51851c0f00f792ea1ce
Sep 10 21:06:47.392432 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.392397    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-tls-cert\") pod \"authorino-744c9c7c76-xpk5f\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") " pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.392529 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.392499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlb8h\" (UniqueName: \"kubernetes.io/projected/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-kube-api-access-dlb8h\") pod \"authorino-744c9c7c76-xpk5f\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") " pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.493678 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.493647    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlb8h\" (UniqueName: \"kubernetes.io/projected/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-kube-api-access-dlb8h\") pod \"authorino-744c9c7c76-xpk5f\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") " pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.493911 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.493730    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-tls-cert\") pod \"authorino-744c9c7c76-xpk5f\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") " pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.496147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.496123    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-tls-cert\") pod \"authorino-744c9c7c76-xpk5f\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") " pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.502282 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.502263    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlb8h\" (UniqueName: \"kubernetes.io/projected/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-kube-api-access-dlb8h\") pod \"authorino-744c9c7c76-xpk5f\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") " pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.603602 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.603528    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:06:47.872999 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:47.872922    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-744c9c7c76-xpk5f"]
Sep 10 21:06:47.875564 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:06:47.875535    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e60e2f3_1ca5_425b_afa0_cf2c166b6581.slice/crio-5bc40e7a7000b2167d4c86e9f0c97602595c3ae427ab1dc08b504b6d6f29d2fa WatchSource:0}: Error finding container 5bc40e7a7000b2167d4c86e9f0c97602595c3ae427ab1dc08b504b6d6f29d2fa: Status 404 returned error can't find the container with id 5bc40e7a7000b2167d4c86e9f0c97602595c3ae427ab1dc08b504b6d6f29d2fa
Sep 10 21:06:48.364407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:48.364364    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wmrt8" event={"ID":"45db3a49-e902-4eeb-b33b-825d23c7597a","Type":"ContainerStarted","Data":"8b56f47494c3e9fb19512073b0344a51631fd77897ddd51851c0f00f792ea1ce"}
Sep 10 21:06:48.365854 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:48.365822    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-744c9c7c76-xpk5f" event={"ID":"2e60e2f3-1ca5-425b-afa0-cf2c166b6581","Type":"ContainerStarted","Data":"5bc40e7a7000b2167d4c86e9f0c97602595c3ae427ab1dc08b504b6d6f29d2fa"}
Sep 10 21:06:49.372791 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.372747    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-wmrt8" podUID="45db3a49-e902-4eeb-b33b-825d23c7597a" containerName="authorino" containerID="cri-o://61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a" gracePeriod=30
Sep 10 21:06:49.373259 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.372749    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wmrt8" event={"ID":"45db3a49-e902-4eeb-b33b-825d23c7597a","Type":"ContainerStarted","Data":"61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a"}
Sep 10 21:06:49.374684 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.374656    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-744c9c7c76-xpk5f" event={"ID":"2e60e2f3-1ca5-425b-afa0-cf2c166b6581","Type":"ContainerStarted","Data":"ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1"}
Sep 10 21:06:49.408782 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.408683    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-wmrt8" podStartSLOduration=1.6307960160000001 podStartE2EDuration="3.408662454s" totalImagesPullingTime="1.777866438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:49.408573373 +0000 UTC m=+503.734334294" watchObservedRunningTime="2026-09-10 21:06:49.408662454 +0000 UTC m=+503.734423380"
Sep 10 21:06:49.434537 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.434482    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-744c9c7c76-xpk5f" podStartSLOduration=1.069667372 podStartE2EDuration="2.434463784s" totalImagesPullingTime="1.364796412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:49.433738409 +0000 UTC m=+503.759499330" watchObservedRunningTime="2026-09-10 21:06:49.434463784 +0000 UTC m=+503.760224727"
Sep 10 21:06:49.610820 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.610792    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wmrt8"
Sep 10 21:06:49.708522 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.708446    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57n62\" (UniqueName: \"kubernetes.io/projected/45db3a49-e902-4eeb-b33b-825d23c7597a-kube-api-access-57n62\") pod \"45db3a49-e902-4eeb-b33b-825d23c7597a\" (UID: \"45db3a49-e902-4eeb-b33b-825d23c7597a\") "
Sep 10 21:06:49.710454 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.710434    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45db3a49-e902-4eeb-b33b-825d23c7597a-kube-api-access-57n62" (OuterVolumeSpecName: "kube-api-access-57n62") pod "45db3a49-e902-4eeb-b33b-825d23c7597a" (UID: "45db3a49-e902-4eeb-b33b-825d23c7597a"). InnerVolumeSpecName "kube-api-access-57n62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:06:49.809054 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:49.809025    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57n62\" (UniqueName: \"kubernetes.io/projected/45db3a49-e902-4eeb-b33b-825d23c7597a-kube-api-access-57n62\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:06:50.380589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.380556    2584 generic.go:356] "Generic (PLEG): container finished" podID="45db3a49-e902-4eeb-b33b-825d23c7597a" containerID="61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a" exitCode=0
Sep 10 21:06:50.381033 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.380593    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wmrt8"
Sep 10 21:06:50.381033 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.380622    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wmrt8" event={"ID":"45db3a49-e902-4eeb-b33b-825d23c7597a","Type":"ContainerDied","Data":"61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a"}
Sep 10 21:06:50.381033 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.380670    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-wmrt8" event={"ID":"45db3a49-e902-4eeb-b33b-825d23c7597a","Type":"ContainerDied","Data":"8b56f47494c3e9fb19512073b0344a51631fd77897ddd51851c0f00f792ea1ce"}
Sep 10 21:06:50.381033 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.380688    2584 scope.go:117] "RemoveContainer" containerID="61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a"
Sep 10 21:06:50.389123 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.389107    2584 scope.go:117] "RemoveContainer" containerID="61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a"
Sep 10 21:06:50.389468 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:50.389445    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a\": container with ID starting with 61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a not found: ID does not exist" containerID="61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a"
Sep 10 21:06:50.389531 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.389477    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a"} err="failed to get container status \"61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a\": rpc error: code = NotFound desc = could not find container \"61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a\": container with ID starting with 61dab9d1912c1e422f1bad336b88b533d44ee1502e64dc31000e9f7218b2964a not found: ID does not exist"
Sep 10 21:06:50.407238 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.407212    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wmrt8"]
Sep 10 21:06:50.409330 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:50.409309    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-wmrt8"]
Sep 10 21:06:52.301741 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:52.301703    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45db3a49-e902-4eeb-b33b-825d23c7597a" path="/var/lib/kubelet/pods/45db3a49-e902-4eeb-b33b-825d23c7597a/volumes"
Sep 10 21:06:54.444254 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.444214    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:54.490772 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.490734    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7769777fc8-x4k7m"]
Sep 10 21:06:54.491098 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.491084    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45db3a49-e902-4eeb-b33b-825d23c7597a" containerName="authorino"
Sep 10 21:06:54.491142 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.491100    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="45db3a49-e902-4eeb-b33b-825d23c7597a" containerName="authorino"
Sep 10 21:06:54.491178 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.491166    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="45db3a49-e902-4eeb-b33b-825d23c7597a" containerName="authorino"
Sep 10 21:06:54.495381 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.495361    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.497548 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.497527    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 21:06:54.497657 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.497530    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 21:06:54.498026 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.498008    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 21:06:54.498406 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.498389    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ddvx9\""
Sep 10 21:06:54.507166 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.507143    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7769777fc8-x4k7m"]
Sep 10 21:06:54.538011 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.537982    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:54.544169 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.544148    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.544229 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.544211    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8tmt\" (UniqueName: \"kubernetes.io/projected/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-kube-api-access-c8tmt\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.544270 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.544233    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.640064 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.640026    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:54.645192 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.645173    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8tmt\" (UniqueName: \"kubernetes.io/projected/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-kube-api-access-c8tmt\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.645253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.645207    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.645253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.645244    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.645393 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.645376    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 10 21:06:54.645443 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.645393    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 21:06:54.645480 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.645467    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs podName:0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad nodeName:}" failed. No retries permitted until 2026-09-10 21:06:55.145449154 +0000 UTC m=+509.471210067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs") pod "maas-controller-7769777fc8-x4k7m" (UID: "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad") : secret "maas-controller-metrics-tls" not found
Sep 10 21:06:54.645525 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.645485    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert podName:0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad nodeName:}" failed. No retries permitted until 2026-09-10 21:06:55.145476002 +0000 UTC m=+509.471236900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert") pod "maas-controller-7769777fc8-x4k7m" (UID: "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad") : secret "maas-controller-webhook-cert" not found
Sep 10 21:06:54.667641 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.667615    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8tmt\" (UniqueName: \"kubernetes.io/projected/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-kube-api-access-c8tmt\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:54.704247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.704185    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-wp48c"]
Sep 10 21:06:54.707399 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.707384    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:54.718709 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.718686    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-wp48c"]
Sep 10 21:06:54.746371 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.746342    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk824\" (UniqueName: \"kubernetes.io/projected/6f53f698-2d72-4f15-8313-38ee72e1e4d1-kube-api-access-tk824\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:54.746549 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.746395    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:54.746549 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.746480    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:54.847818 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.847779    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:54.847979 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.847865    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk824\" (UniqueName: \"kubernetes.io/projected/6f53f698-2d72-4f15-8313-38ee72e1e4d1-kube-api-access-tk824\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:54.847979 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.847908    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:54.847979 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.847924    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 21:06:54.848074 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.847984    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 10 21:06:54.848074 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.847989    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs podName:6f53f698-2d72-4f15-8313-38ee72e1e4d1 nodeName:}" failed. No retries permitted until 2026-09-10 21:06:55.347975236 +0000 UTC m=+509.673736140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs") pod "maas-controller-76f4d89c9c-wp48c" (UID: "6f53f698-2d72-4f15-8313-38ee72e1e4d1") : secret "maas-controller-metrics-tls" not found
Sep 10 21:06:54.848074 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:54.848026    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert podName:6f53f698-2d72-4f15-8313-38ee72e1e4d1 nodeName:}" failed. No retries permitted until 2026-09-10 21:06:55.348013698 +0000 UTC m=+509.673774597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert") pod "maas-controller-76f4d89c9c-wp48c" (UID: "6f53f698-2d72-4f15-8313-38ee72e1e4d1") : secret "maas-controller-webhook-cert" not found
Sep 10 21:06:54.857849 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:54.857819    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk824\" (UniqueName: \"kubernetes.io/projected/6f53f698-2d72-4f15-8313-38ee72e1e4d1-kube-api-access-tk824\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:55.051238 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.051159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:55.150451 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.150401    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:55.150648 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.150538    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:55.150648 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:55.150629    2584 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 10 21:06:55.150746 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:06:55.150687    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert podName:0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad nodeName:}" failed. No retries permitted until 2026-09-10 21:06:56.150672744 +0000 UTC m=+510.476433645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert") pod "maas-controller-7769777fc8-x4k7m" (UID: "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad") : secret "maas-controller-webhook-cert" not found
Sep 10 21:06:55.152898 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.152874    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:55.348757 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.348680    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:55.352170 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.352140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:55.352330 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.352194    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:55.354548 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.354521    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:55.354672 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.354607    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs\") pod \"maas-controller-76f4d89c9c-wp48c\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") " pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:55.618482 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.618154    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:55.891306 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:55.891277    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-wp48c"]
Sep 10 21:06:55.892725 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:06:55.892692    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f53f698_2d72_4f15_8313_38ee72e1e4d1.slice/crio-185c367f7daeda4b9f71b84d3f39a5f75835ba1e3a321d6c300029311b625fef WatchSource:0}: Error finding container 185c367f7daeda4b9f71b84d3f39a5f75835ba1e3a321d6c300029311b625fef: Status 404 returned error can't find the container with id 185c367f7daeda4b9f71b84d3f39a5f75835ba1e3a321d6c300029311b625fef
Sep 10 21:06:56.157394 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:56.157367    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:56.159754 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:56.159732    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert\") pod \"maas-controller-7769777fc8-x4k7m\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") " pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:56.305273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:56.305244    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:56.415227 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:56.415150    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-wp48c" event={"ID":"6f53f698-2d72-4f15-8313-38ee72e1e4d1","Type":"ContainerStarted","Data":"185c367f7daeda4b9f71b84d3f39a5f75835ba1e3a321d6c300029311b625fef"}
Sep 10 21:06:56.583089 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:56.583062    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7769777fc8-x4k7m"]
Sep 10 21:06:56.585002 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:06:56.584975    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3ad0d3_a9bc_4126_8e32_fac4e1cecaad.slice/crio-e2a40902a89ee63d17c251f3fe5ffa523f65d0e8e2306c9b1cb7e9298bde3aa0 WatchSource:0}: Error finding container e2a40902a89ee63d17c251f3fe5ffa523f65d0e8e2306c9b1cb7e9298bde3aa0: Status 404 returned error can't find the container with id e2a40902a89ee63d17c251f3fe5ffa523f65d0e8e2306c9b1cb7e9298bde3aa0
Sep 10 21:06:57.424030 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:57.423986    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-x4k7m" event={"ID":"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad","Type":"ContainerStarted","Data":"e2a40902a89ee63d17c251f3fe5ffa523f65d0e8e2306c9b1cb7e9298bde3aa0"}
Sep 10 21:06:58.796029 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:58.795993    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:58.904809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:58.904772    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:59.435933 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:59.435889    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-x4k7m" event={"ID":"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad","Type":"ContainerStarted","Data":"235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06"}
Sep 10 21:06:59.436134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:59.436041    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:06:59.437350 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:59.437329    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-wp48c" event={"ID":"6f53f698-2d72-4f15-8313-38ee72e1e4d1","Type":"ContainerStarted","Data":"a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb"}
Sep 10 21:06:59.437490 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:59.437466    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:06:59.467530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:59.467481    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7769777fc8-x4k7m" podStartSLOduration=3.470651156 podStartE2EDuration="5.46746627s" totalImagesPullingTime="1.996815114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:59.465850336 +0000 UTC m=+513.791611256" watchObservedRunningTime="2026-09-10 21:06:59.46746627 +0000 UTC m=+513.793227190"
Sep 10 21:06:59.528264 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:06:59.528199    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f4d89c9c-wp48c" podStartSLOduration=2.845757781 podStartE2EDuration="5.528177223s" totalImagesPullingTime="2.682419442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:59.527086385 +0000 UTC m=+513.852847308" watchObservedRunningTime="2026-09-10 21:06:59.528177223 +0000 UTC m=+513.853938148"
Sep 10 21:07:10.444353 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.444321    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:07:10.444829 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.444463    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:07:10.527619 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.527581    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7769777fc8-x4k7m"]
Sep 10 21:07:10.528037 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.527978    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7769777fc8-x4k7m" podUID="0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" containerName="manager" containerID="cri-o://235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06" gracePeriod=30
Sep 10 21:07:10.773371 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.773347    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:07:10.889545 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.889510    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert\") pod \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") "
Sep 10 21:07:10.889704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.889558    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8tmt\" (UniqueName: \"kubernetes.io/projected/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-kube-api-access-c8tmt\") pod \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") "
Sep 10 21:07:10.889704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.889582    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs\") pod \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\" (UID: \"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad\") "
Sep 10 21:07:10.891776 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.891736    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" (UID: "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:07:10.891776 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.891756    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" (UID: "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:07:10.891776 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.891771    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-kube-api-access-c8tmt" (OuterVolumeSpecName: "kube-api-access-c8tmt") pod "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" (UID: "0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad"). InnerVolumeSpecName "kube-api-access-c8tmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:07:10.990961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.990882    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-webhook-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:07:10.990961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.990908    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8tmt\" (UniqueName: \"kubernetes.io/projected/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-kube-api-access-c8tmt\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:07:10.990961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:10.990919    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:07:11.493999 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.493966    2584 generic.go:356] "Generic (PLEG): container finished" podID="0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" containerID="235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06" exitCode=0
Sep 10 21:07:11.494582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.494007    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7769777fc8-x4k7m"
Sep 10 21:07:11.494582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.494050    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-x4k7m" event={"ID":"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad","Type":"ContainerDied","Data":"235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06"}
Sep 10 21:07:11.494582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.494102    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7769777fc8-x4k7m" event={"ID":"0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad","Type":"ContainerDied","Data":"e2a40902a89ee63d17c251f3fe5ffa523f65d0e8e2306c9b1cb7e9298bde3aa0"}
Sep 10 21:07:11.494582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.494123    2584 scope.go:117] "RemoveContainer" containerID="235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06"
Sep 10 21:07:11.502973 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.502958    2584 scope.go:117] "RemoveContainer" containerID="235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06"
Sep 10 21:07:11.503244 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:07:11.503222    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06\": container with ID starting with 235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06 not found: ID does not exist" containerID="235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06"
Sep 10 21:07:11.503317 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.503254    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06"} err="failed to get container status \"235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06\": rpc error: code = NotFound desc = could not find container \"235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06\": container with ID starting with 235bdf32973e2919598c550f87fcc65759aa22cceeeb9dfc590adf0ee00cfc06 not found: ID does not exist"
Sep 10 21:07:11.519878 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.519855    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7769777fc8-x4k7m"]
Sep 10 21:07:11.528719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:11.528691    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7769777fc8-x4k7m"]
Sep 10 21:07:12.301222 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:12.301187    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" path="/var/lib/kubelet/pods/0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad/volumes"
Sep 10 21:07:16.460727 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:16.460691    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:30.570476 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:30.570411    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:30.674962 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:30.674913    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:32.002750 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.002698    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:32.100400 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.100370    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:32.127184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.127146    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"]
Sep 10 21:07:32.127581 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.127562    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" containerName="manager"
Sep 10 21:07:32.127728 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.127583    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" containerName="manager"
Sep 10 21:07:32.127728 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.127697    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d3ad0d3-a9bc-4126-8e32-fac4e1cecaad" containerName="manager"
Sep 10 21:07:32.134606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.134586    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.136999 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.136977    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:07:32.137134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.136963    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 21:07:32.137247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.137236    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:07:32.146402 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.146383    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"]
Sep 10 21:07:32.278664 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.278579    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk4n5\" (UniqueName: \"kubernetes.io/projected/964998e7-4388-4a6b-9e30-6a70ea7620c9-kube-api-access-sk4n5\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.278814 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.278671    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.278814 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.278741    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964998e7-4388-4a6b-9e30-6a70ea7620c9-openshift-service-ca\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.278814 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.278810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.380171 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.380136    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.380171 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.380175    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk4n5\" (UniqueName: \"kubernetes.io/projected/964998e7-4388-4a6b-9e30-6a70ea7620c9-kube-api-access-sk4n5\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.380398 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.380250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.380398 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.380286    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964998e7-4388-4a6b-9e30-6a70ea7620c9-openshift-service-ca\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.380398 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:07:32.380297    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 21:07:32.380398 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:07:32.380378    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 21:07:32.380627 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:07:32.380385    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs podName:964998e7-4388-4a6b-9e30-6a70ea7620c9 nodeName:}" failed. No retries permitted until 2026-09-10 21:07:32.880353341 +0000 UTC m=+547.206114250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs") pod "maas-api-7f7f8945f9-hfwsn" (UID: "964998e7-4388-4a6b-9e30-6a70ea7620c9") : secret "maas-api-metrics-tls" not found
Sep 10 21:07:32.380627 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:07:32.380489    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls podName:964998e7-4388-4a6b-9e30-6a70ea7620c9 nodeName:}" failed. No retries permitted until 2026-09-10 21:07:32.8804733 +0000 UTC m=+547.206234200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls") pod "maas-api-7f7f8945f9-hfwsn" (UID: "964998e7-4388-4a6b-9e30-6a70ea7620c9") : secret "maas-api-serving-cert" not found
Sep 10 21:07:32.380961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.380941    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964998e7-4388-4a6b-9e30-6a70ea7620c9-openshift-service-ca\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.391871 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.391838    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk4n5\" (UniqueName: \"kubernetes.io/projected/964998e7-4388-4a6b-9e30-6a70ea7620c9-kube-api-access-sk4n5\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.454351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.454304    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:32.553362 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.553261    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:32.654026 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.653993    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"]
Sep 10 21:07:32.656138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.656106    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:32.657660 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.657641    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:32.660356 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.660317    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 21:07:32.669375 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.669353    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"]
Sep 10 21:07:32.782795 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.782759    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f450a502-4479-474c-a9b3-c403c58f3f3b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ttz9p\" (UID: \"f450a502-4479-474c-a9b3-c403c58f3f3b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:32.782795 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.782794    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fp8n\" (UniqueName: \"kubernetes.io/projected/f450a502-4479-474c-a9b3-c403c58f3f3b-kube-api-access-4fp8n\") pod \"payload-pre-processing-59fbc47bf8-ttz9p\" (UID: \"f450a502-4479-474c-a9b3-c403c58f3f3b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:32.884315 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.884230    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f450a502-4479-474c-a9b3-c403c58f3f3b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ttz9p\" (UID: \"f450a502-4479-474c-a9b3-c403c58f3f3b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:32.884315 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.884283    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fp8n\" (UniqueName: \"kubernetes.io/projected/f450a502-4479-474c-a9b3-c403c58f3f3b-kube-api-access-4fp8n\") pod \"payload-pre-processing-59fbc47bf8-ttz9p\" (UID: \"f450a502-4479-474c-a9b3-c403c58f3f3b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:32.884315 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.884316    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.884564 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.884381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.885099 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.885079    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f450a502-4479-474c-a9b3-c403c58f3f3b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ttz9p\" (UID: \"f450a502-4479-474c-a9b3-c403c58f3f3b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:32.887016 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.886991    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.887136 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.887044    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls\") pod \"maas-api-7f7f8945f9-hfwsn\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:32.894777 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.894755    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fp8n\" (UniqueName: \"kubernetes.io/projected/f450a502-4479-474c-a9b3-c403c58f3f3b-kube-api-access-4fp8n\") pod \"payload-pre-processing-59fbc47bf8-ttz9p\" (UID: \"f450a502-4479-474c-a9b3-c403c58f3f3b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:32.968080 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:32.968045    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:33.045100 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:33.044746    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:33.251312 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:33.251279    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"]
Sep 10 21:07:33.255383 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:07:33.255352    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf450a502_4479_474c_a9b3_c403c58f3f3b.slice/crio-61d7fcb77be04806df560dc27308a3ea6961b922fa600aff3f4e1051ee655249 WatchSource:0}: Error finding container 61d7fcb77be04806df560dc27308a3ea6961b922fa600aff3f4e1051ee655249: Status 404 returned error can't find the container with id 61d7fcb77be04806df560dc27308a3ea6961b922fa600aff3f4e1051ee655249
Sep 10 21:07:33.342768 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:33.342582    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"]
Sep 10 21:07:33.345599 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:07:33.345570    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964998e7_4388_4a6b_9e30_6a70ea7620c9.slice/crio-16eb7666fa81d7babdd12888d39445836ae96fc93c86816f2d2619426e96a35e WatchSource:0}: Error finding container 16eb7666fa81d7babdd12888d39445836ae96fc93c86816f2d2619426e96a35e: Status 404 returned error can't find the container with id 16eb7666fa81d7babdd12888d39445836ae96fc93c86816f2d2619426e96a35e
Sep 10 21:07:33.600655 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:33.600554    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p" event={"ID":"f450a502-4479-474c-a9b3-c403c58f3f3b","Type":"ContainerStarted","Data":"61d7fcb77be04806df560dc27308a3ea6961b922fa600aff3f4e1051ee655249"}
Sep 10 21:07:33.601693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:33.601669    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn" event={"ID":"964998e7-4388-4a6b-9e30-6a70ea7620c9","Type":"ContainerStarted","Data":"16eb7666fa81d7babdd12888d39445836ae96fc93c86816f2d2619426e96a35e"}
Sep 10 21:07:36.620504 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:36.620464    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p" event={"ID":"f450a502-4479-474c-a9b3-c403c58f3f3b","Type":"ContainerStarted","Data":"3e9181231e61be0d7e4efd785dd1143cf17ea119a86e8d462d3147054da66dcf"}
Sep 10 21:07:36.620920 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:36.620541    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:36.621918 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:36.621894    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn" event={"ID":"964998e7-4388-4a6b-9e30-6a70ea7620c9","Type":"ContainerStarted","Data":"12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085"}
Sep 10 21:07:36.622050 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:36.622034    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:36.641917 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:36.641872    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p" podStartSLOduration=1.9101672010000001 podStartE2EDuration="4.641854087s" totalImagesPullingTime="2.731686886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:07:36.641438327 +0000 UTC m=+550.967199241" watchObservedRunningTime="2026-09-10 21:07:36.641854087 +0000 UTC m=+550.967615008"
Sep 10 21:07:36.669639 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:36.669593    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn" podStartSLOduration=2.028202455 podStartE2EDuration="4.669577772s" totalImagesPullingTime="2.641375317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:07:36.667607691 +0000 UTC m=+550.993368611" watchObservedRunningTime="2026-09-10 21:07:36.669577772 +0000 UTC m=+550.995338693"
Sep 10 21:07:42.634263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:42.634234    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:07:43.640224 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:43.640184    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:43.732837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:43.732803    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:47.631005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:47.630973    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ttz9p"
Sep 10 21:07:50.369619 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:50.369566    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:59.700904 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:07:59.700866    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:09.532299 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.532255    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"]
Sep 10 21:08:09.539759 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.539733    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.542339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.542319    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 21:08:09.543367 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.543349    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-njkdt\""
Sep 10 21:08:09.543471 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.543402    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 21:08:09.543471 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.543406    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 21:08:09.547337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.547319    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"]
Sep 10 21:08:09.576912 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.576881    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.577051 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.576917    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.577051 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.576941    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.577131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.577046    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhdf\" (UniqueName: \"kubernetes.io/projected/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-kube-api-access-9nhdf\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.577131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.577109    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.577211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.577135    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.635869 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.635820    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:09.678168 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678133    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678185    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678232    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678261    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhdf\" (UniqueName: \"kubernetes.io/projected/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-kube-api-access-9nhdf\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678325    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678617 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678595    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678645    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.678814 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.678794    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.680594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.680570    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.680879 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.680860    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.686870 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.686845    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhdf\" (UniqueName: \"kubernetes.io/projected/5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904-kube-api-access-9nhdf\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h\" (UID: \"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:09.851212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:09.851126    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:10.128823 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:10.128711    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"]
Sep 10 21:08:10.131651 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:08:10.131615    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a49c2dc_ff84_4d40_9cfb_b7c54e4ef904.slice/crio-0cad9a39268ffec6f46e4b2e3bee6cc37dbabf9e7e183d3718f71026902bb6e2 WatchSource:0}: Error finding container 0cad9a39268ffec6f46e4b2e3bee6cc37dbabf9e7e183d3718f71026902bb6e2: Status 404 returned error can't find the container with id 0cad9a39268ffec6f46e4b2e3bee6cc37dbabf9e7e183d3718f71026902bb6e2
Sep 10 21:08:10.779326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:10.779282    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h" event={"ID":"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904","Type":"ContainerStarted","Data":"0cad9a39268ffec6f46e4b2e3bee6cc37dbabf9e7e183d3718f71026902bb6e2"}
Sep 10 21:08:11.785085 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:11.785041    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h" event={"ID":"5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904","Type":"ContainerStarted","Data":"d69e6e9ca646a70e2a6f96bc4981a98509f074d13acb6ceb68c8ff2c13b84a50"}
Sep 10 21:08:13.076024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:13.075984    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:19.851935 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:19.851902    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:19.852506 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:19.852193    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:19.864847 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:19.864822    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:19.887874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:19.887820    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h" podStartSLOduration=9.810058075 podStartE2EDuration="10.887807224s" totalImagesPullingTime="1.077749149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:08:11.810446913 +0000 UTC m=+586.136207836" watchObservedRunningTime="2026-09-10 21:08:19.887807224 +0000 UTC m=+594.213568145"
Sep 10 21:08:20.839454 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:20.839407    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h"
Sep 10 21:08:21.037977 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.037941    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"]
Sep 10 21:08:21.040517 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.040497    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.043552 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.043530    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 21:08:21.056625 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.056602    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"]
Sep 10 21:08:21.090537 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.090467    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:21.183170 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.183134    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.183339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.183185    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.183339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.183204    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.183339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.183226    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.183339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.183317    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.183515 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.183339    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dckg\" (UniqueName: \"kubernetes.io/projected/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-kube-api-access-4dckg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.284410 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.284376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.284620 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.284435    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dckg\" (UniqueName: \"kubernetes.io/projected/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-kube-api-access-4dckg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.284620 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.284483    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.284620 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.284519    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.284620 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.284538    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.284620 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.284568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.285066 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.285039    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.285066 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.285061    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.285257 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.285127    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.287049 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.287013    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.287274 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.287256    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.295139 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.295115    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dckg\" (UniqueName: \"kubernetes.io/projected/c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2-kube-api-access-4dckg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl\" (UID: \"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.352544 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.352477    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:21.632513 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.632485    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"]
Sep 10 21:08:21.635148 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:08:21.635114    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9fb2ff4_5cbc_44ee_b41a_ca2d3750f6c2.slice/crio-9a1717597c2858c1dc2dc1c5976c91079b341123eb96a0c4729c5b580a42ffbe WatchSource:0}: Error finding container 9a1717597c2858c1dc2dc1c5976c91079b341123eb96a0c4729c5b580a42ffbe: Status 404 returned error can't find the container with id 9a1717597c2858c1dc2dc1c5976c91079b341123eb96a0c4729c5b580a42ffbe
Sep 10 21:08:21.834106 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:21.834080    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl" event={"ID":"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2","Type":"ContainerStarted","Data":"9a1717597c2858c1dc2dc1c5976c91079b341123eb96a0c4729c5b580a42ffbe"}
Sep 10 21:08:22.840745 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:22.840683    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl" event={"ID":"c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2","Type":"ContainerStarted","Data":"f5adac89ec77a00791bfdfd5c8473fd7b923f39b94fcdf9072dec1767919c34a"}
Sep 10 21:08:22.865612 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:22.865556    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl" podStartSLOduration=1.67689852 podStartE2EDuration="1.865539588s" totalImagesPullingTime="188.641068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:08:22.862817073 +0000 UTC m=+597.188577993" watchObservedRunningTime="2026-09-10 21:08:22.865539588 +0000 UTC m=+597.191300545"
Sep 10 21:08:23.028321 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:23.028284    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:26.124953 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:26.124927    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:08:26.126781 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:26.126761    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:08:31.353339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:31.353303    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:31.354654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:31.353374    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:31.366199 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:31.366173    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:31.899548 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:31.899517    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl"
Sep 10 21:08:40.681215 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:40.681179    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:42.877686 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:42.877644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:49.068602 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:49.068510    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:59.580955 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:59.580919    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:59.680585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:08:59.680549    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:40.658496 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:40.658458    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:43.951902 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:43.951867    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-744c9c7c76-xpk5f"]
Sep 10 21:09:43.952315 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:43.952094    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-744c9c7c76-xpk5f" podUID="2e60e2f3-1ca5-425b-afa0-cf2c166b6581" containerName="authorino" containerID="cri-o://ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1" gracePeriod=30
Sep 10 21:09:44.212624 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.212569    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:09:44.231020 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.230993    2584 generic.go:356] "Generic (PLEG): container finished" podID="2e60e2f3-1ca5-425b-afa0-cf2c166b6581" containerID="ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1" exitCode=0
Sep 10 21:09:44.231145 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.231028    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-744c9c7c76-xpk5f"
Sep 10 21:09:44.231145 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.231072    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-744c9c7c76-xpk5f" event={"ID":"2e60e2f3-1ca5-425b-afa0-cf2c166b6581","Type":"ContainerDied","Data":"ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1"}
Sep 10 21:09:44.231145 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.231123    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-744c9c7c76-xpk5f" event={"ID":"2e60e2f3-1ca5-425b-afa0-cf2c166b6581","Type":"ContainerDied","Data":"5bc40e7a7000b2167d4c86e9f0c97602595c3ae427ab1dc08b504b6d6f29d2fa"}
Sep 10 21:09:44.231145 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.231136    2584 scope.go:117] "RemoveContainer" containerID="ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1"
Sep 10 21:09:44.240232 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.240213    2584 scope.go:117] "RemoveContainer" containerID="ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1"
Sep 10 21:09:44.240611 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:09:44.240574    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1\": container with ID starting with ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1 not found: ID does not exist" containerID="ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1"
Sep 10 21:09:44.240716 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.240622    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1"} err="failed to get container status \"ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1\": rpc error: code = NotFound desc = could not find container \"ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1\": container with ID starting with ea0c144cd91193a7aefe7d7f027642802c4f2b78470fcc7f99af8b0ebafe41f1 not found: ID does not exist"
Sep 10 21:09:44.325473 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.325444    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-tls-cert\") pod \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") "
Sep 10 21:09:44.325616 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.325494    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlb8h\" (UniqueName: \"kubernetes.io/projected/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-kube-api-access-dlb8h\") pod \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\" (UID: \"2e60e2f3-1ca5-425b-afa0-cf2c166b6581\") "
Sep 10 21:09:44.327446 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.327388    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-kube-api-access-dlb8h" (OuterVolumeSpecName: "kube-api-access-dlb8h") pod "2e60e2f3-1ca5-425b-afa0-cf2c166b6581" (UID: "2e60e2f3-1ca5-425b-afa0-cf2c166b6581"). InnerVolumeSpecName "kube-api-access-dlb8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:09:44.334886 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.334862    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2e60e2f3-1ca5-425b-afa0-cf2c166b6581" (UID: "2e60e2f3-1ca5-425b-afa0-cf2c166b6581"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:09:44.426957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.426929    2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-tls-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:09:44.426957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.426955    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlb8h\" (UniqueName: \"kubernetes.io/projected/2e60e2f3-1ca5-425b-afa0-cf2c166b6581-kube-api-access-dlb8h\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:09:44.450518 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.450487    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:44.552806 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.552775    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-744c9c7c76-xpk5f"]
Sep 10 21:09:44.554940 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.554911    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:44.558580 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:44.558559    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-744c9c7c76-xpk5f"]
Sep 10 21:09:45.549029 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:45.548991    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:45.648381 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:45.648341    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:46.301333 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:46.301303    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e60e2f3-1ca5-425b-afa0-cf2c166b6581" path="/var/lib/kubelet/pods/2e60e2f3-1ca5-425b-afa0-cf2c166b6581/volumes"
Sep 10 21:09:47.284206 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:47.284159    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:47.487347 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:09:47.487293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:00.672869 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:00.672828    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:00.774515 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:00.774480    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:01.104476 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:01.104383    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:01.202663 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:01.202620    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:02.045846 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:02.045807    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:02.143325 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:02.143290    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:02.467809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:02.467773    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:04.060705 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:04.060672    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:04.158274 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:04.158238    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:05.180380 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.180342    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"]
Sep 10 21:10:05.180945 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.180926    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e60e2f3-1ca5-425b-afa0-cf2c166b6581" containerName="authorino"
Sep 10 21:10:05.180990 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.180948    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e60e2f3-1ca5-425b-afa0-cf2c166b6581" containerName="authorino"
Sep 10 21:10:05.181058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.181048    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e60e2f3-1ca5-425b-afa0-cf2c166b6581" containerName="authorino"
Sep 10 21:10:05.183124 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.183107    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.185568 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.185551    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-76b274-serving-cert\""
Sep 10 21:10:05.195572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.195539    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"]
Sep 10 21:10:05.209204 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.209179    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-metrics-certs\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.209343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.209241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.209343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.209278    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmfx4\" (UniqueName: \"kubernetes.io/projected/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-kube-api-access-lmfx4\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.209343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.209297    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-openshift-service-ca\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.219471 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.219438    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:05.309854 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.309813    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.310021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.309866    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmfx4\" (UniqueName: \"kubernetes.io/projected/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-kube-api-access-lmfx4\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.310021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.309886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-openshift-service-ca\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.310021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.309921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-metrics-certs\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.310021 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:10:05.309980    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-serving-cert: secret "maas-api-e2e-inf-a-76b274-serving-cert" not found
Sep 10 21:10:05.310233 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:10:05.310068    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls podName:6f70ac02-5e43-44dc-8b8d-c93f7c801c94 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:05.810046768 +0000 UTC m=+700.135807672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls") pod "maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj" (UID: "6f70ac02-5e43-44dc-8b8d-c93f7c801c94") : secret "maas-api-e2e-inf-a-76b274-serving-cert" not found
Sep 10 21:10:05.310643 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.310622    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-openshift-service-ca\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.312260 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.312234    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-metrics-certs\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.319200 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.319181    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmfx4\" (UniqueName: \"kubernetes.io/projected/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-kube-api-access-lmfx4\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.323511 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.323480    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:05.422113 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.422073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:05.719598 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.719559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:05.814600 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.814571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.817057 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.817030    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls\") pod \"maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:05.824471 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.824442    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"]
Sep 10 21:10:05.824875 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.824842    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:05.827015 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.827000    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:05.830442 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.830410    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-76b274\""
Sep 10 21:10:05.841343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.841317    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"]
Sep 10 21:10:05.864251 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.864222    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"]
Sep 10 21:10:05.866511 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.866492    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:05.868821 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.868799    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-76b274-dockercfg-m84fr\""
Sep 10 21:10:05.878328 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.878305    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"]
Sep 10 21:10:05.915299 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.915274    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ebc320-d1df-4fbe-8a91-18297bf05234-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:05.915475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.915306    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7fbc\" (UniqueName: \"kubernetes.io/projected/045fa49f-6f8a-444c-9626-aa66ced692a0-kube-api-access-m7fbc\") pod \"payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:05.915475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.915329    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgvdt\" (UniqueName: \"kubernetes.io/projected/f6ebc320-d1df-4fbe-8a91-18297bf05234-kube-api-access-jgvdt\") pod \"payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:05.915475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:05.915376    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/045fa49f-6f8a-444c-9626-aa66ced692a0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:06.015938 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.015842    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgvdt\" (UniqueName: \"kubernetes.io/projected/f6ebc320-d1df-4fbe-8a91-18297bf05234-kube-api-access-jgvdt\") pod \"payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:06.015938 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.015883    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/045fa49f-6f8a-444c-9626-aa66ced692a0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:06.016174 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.016027    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ebc320-d1df-4fbe-8a91-18297bf05234-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:06.016174 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.016054    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7fbc\" (UniqueName: \"kubernetes.io/projected/045fa49f-6f8a-444c-9626-aa66ced692a0-kube-api-access-m7fbc\") pod \"payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:06.016884 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.016857    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/045fa49f-6f8a-444c-9626-aa66ced692a0-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:06.016950 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.016862    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ebc320-d1df-4fbe-8a91-18297bf05234-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:06.024632 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.024602    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgvdt\" (UniqueName: \"kubernetes.io/projected/f6ebc320-d1df-4fbe-8a91-18297bf05234-kube-api-access-jgvdt\") pod \"payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:06.024936 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.024918    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7fbc\" (UniqueName: \"kubernetes.io/projected/045fa49f-6f8a-444c-9626-aa66ced692a0-kube-api-access-m7fbc\") pod \"payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") " pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:06.097868 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.097825    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:06.137458 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.137414    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:06.179060 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.178852    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:06.367660 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.367580    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"]
Sep 10 21:10:06.371645 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:06.371120    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f70ac02_5e43_44dc_8b8d_c93f7c801c94.slice/crio-8f11e9a129f89fd7c99fc26fc8c61c984801ec5b8bec11ce52fb2a00478fb693 WatchSource:0}: Error finding container 8f11e9a129f89fd7c99fc26fc8c61c984801ec5b8bec11ce52fb2a00478fb693: Status 404 returned error can't find the container with id 8f11e9a129f89fd7c99fc26fc8c61c984801ec5b8bec11ce52fb2a00478fb693
Sep 10 21:10:06.373954 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.373931    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:10:06.421685 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.421655    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"]
Sep 10 21:10:06.424505 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:06.424454    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ebc320_d1df_4fbe_8a91_18297bf05234.slice/crio-c3762c0f14cade4ea77705f2e228665e789385e71c6e8ab7033a5c7965da0b0e WatchSource:0}: Error finding container c3762c0f14cade4ea77705f2e228665e789385e71c6e8ab7033a5c7965da0b0e: Status 404 returned error can't find the container with id c3762c0f14cade4ea77705f2e228665e789385e71c6e8ab7033a5c7965da0b0e
Sep 10 21:10:06.459833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:06.459798    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"]
Sep 10 21:10:06.462622 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:06.462596    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045fa49f_6f8a_444c_9626_aa66ced692a0.slice/crio-4f81f35cf1d982da88ad4ad1103c69ee0c3ccb68082ee1c7de93b14b07b0a552 WatchSource:0}: Error finding container 4f81f35cf1d982da88ad4ad1103c69ee0c3ccb68082ee1c7de93b14b07b0a552: Status 404 returned error can't find the container with id 4f81f35cf1d982da88ad4ad1103c69ee0c3ccb68082ee1c7de93b14b07b0a552
Sep 10 21:10:07.357712 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.357679    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb" event={"ID":"045fa49f-6f8a-444c-9626-aa66ced692a0","Type":"ContainerStarted","Data":"90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b"}
Sep 10 21:10:07.357907 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.357736    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb" event={"ID":"045fa49f-6f8a-444c-9626-aa66ced692a0","Type":"ContainerStarted","Data":"4f81f35cf1d982da88ad4ad1103c69ee0c3ccb68082ee1c7de93b14b07b0a552"}
Sep 10 21:10:07.357907 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.357845    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:07.359180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.359156    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7" event={"ID":"f6ebc320-d1df-4fbe-8a91-18297bf05234","Type":"ContainerStarted","Data":"d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3"}
Sep 10 21:10:07.359255 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.359205    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7" event={"ID":"f6ebc320-d1df-4fbe-8a91-18297bf05234","Type":"ContainerStarted","Data":"c3762c0f14cade4ea77705f2e228665e789385e71c6e8ab7033a5c7965da0b0e"}
Sep 10 21:10:07.359255 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.359228    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:07.360769 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.360747    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj" event={"ID":"6f70ac02-5e43-44dc-8b8d-c93f7c801c94","Type":"ContainerStarted","Data":"3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c"}
Sep 10 21:10:07.360870 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.360781    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj" event={"ID":"6f70ac02-5e43-44dc-8b8d-c93f7c801c94","Type":"ContainerStarted","Data":"8f11e9a129f89fd7c99fc26fc8c61c984801ec5b8bec11ce52fb2a00478fb693"}
Sep 10 21:10:07.360918 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.360883    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:07.376941 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.376881    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb" podStartSLOduration=2.376863454 podStartE2EDuration="2.376863454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:07.373870833 +0000 UTC m=+701.699631755" watchObservedRunningTime="2026-09-10 21:10:07.376863454 +0000 UTC m=+701.702624375"
Sep 10 21:10:07.393170 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.393120    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj" podStartSLOduration=1.8750670139999999 podStartE2EDuration="2.393083495s" totalImagesPullingTime="518.016481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:07.390802402 +0000 UTC m=+701.716563320" watchObservedRunningTime="2026-09-10 21:10:07.393083495 +0000 UTC m=+701.718844419"
Sep 10 21:10:07.405731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:07.405689    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7" podStartSLOduration=2.405674957 podStartE2EDuration="2.405674957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:07.40522648 +0000 UTC m=+701.730987401" watchObservedRunningTime="2026-09-10 21:10:07.405674957 +0000 UTC m=+701.731435876"
Sep 10 21:10:08.860501 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:08.860454    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:08.961540 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:08.961508    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:09.083430 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:09.083391    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:09.384868 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:09.384830    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:09.512223 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:09.512194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:10.151586 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:10.151536    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:10.460516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:10.460384    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:10.574532 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:10.574498    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:11.631566 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:11.631511    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:11.715279 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:11.715245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:12.415679 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.415644    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"]
Sep 10 21:10:12.420289 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.420267    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.422647 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.422627    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-75ed5f65-kserve-self-signed-certs\""
Sep 10 21:10:12.429766 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.429742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"]
Sep 10 21:10:12.478486 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.478455    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-model-cache\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.478658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.478545    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-dshm\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.478658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.478572    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b6e999e-873d-43a0-86e4-87289a59863c-tls-certs\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.478658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.478589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqpxz\" (UniqueName: \"kubernetes.io/projected/4b6e999e-873d-43a0-86e4-87289a59863c-kube-api-access-lqpxz\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.478658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.478605    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-home\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.478658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.478620    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-tmp-dir\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.579712 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.579677    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-model-cache\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.579921 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.579767    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-dshm\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.579921 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.579795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b6e999e-873d-43a0-86e4-87289a59863c-tls-certs\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.579921 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.579810    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqpxz\" (UniqueName: \"kubernetes.io/projected/4b6e999e-873d-43a0-86e4-87289a59863c-kube-api-access-lqpxz\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.579921 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.579827    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-home\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.579921 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.579842    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-tmp-dir\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.580240 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.580218    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-tmp-dir\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.580240 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.580230    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-home\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.580361 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.580342    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-model-cache\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.582133 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.582106    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-dshm\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.582345 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.582325    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b6e999e-873d-43a0-86e4-87289a59863c-tls-certs\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.589134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.589104    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqpxz\" (UniqueName: \"kubernetes.io/projected/4b6e999e-873d-43a0-86e4-87289a59863c-kube-api-access-lqpxz\") pod \"e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") " pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.720586 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.720500    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:12.730270 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.730237    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:12.972481 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:12.972338    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:13.011075 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.011001    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"]
Sep 10 21:10:13.013494 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:13.013465    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b6e999e_873d_43a0_86e4_87289a59863c.slice/crio-c443100c0c3b2d3d2b7225db819707033a6fafdf8512321d3ecfacbc65509532 WatchSource:0}: Error finding container c443100c0c3b2d3d2b7225db819707033a6fafdf8512321d3ecfacbc65509532: Status 404 returned error can't find the container with id c443100c0c3b2d3d2b7225db819707033a6fafdf8512321d3ecfacbc65509532
Sep 10 21:10:13.078441 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.078389    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:13.172843 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.172805    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:13.273489 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.273377    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:13.371109 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.371079    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:10:13.395973 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.395938    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6" event={"ID":"4b6e999e-873d-43a0-86e4-87289a59863c","Type":"ContainerStarted","Data":"b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1"}
Sep 10 21:10:13.395973 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.395982    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6" event={"ID":"4b6e999e-873d-43a0-86e4-87289a59863c","Type":"ContainerStarted","Data":"c443100c0c3b2d3d2b7225db819707033a6fafdf8512321d3ecfacbc65509532"}
Sep 10 21:10:13.419897 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.419846    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6" podStartSLOduration=1.419827798 podStartE2EDuration="1.419827798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:13.41892022 +0000 UTC m=+707.744681143" watchObservedRunningTime="2026-09-10 21:10:13.419827798 +0000 UTC m=+707.745588719"
Sep 10 21:10:13.614905 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.614829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:13.718757 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.718719    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:13.727694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.727662    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"]
Sep 10 21:10:13.731129 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.731111    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:13.733564 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.733545    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d9fd5e6e\""
Sep 10 21:10:13.740625 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.740605    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"]
Sep 10 21:10:13.790408 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.790372    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1069ab2-ad70-48ec-8a8e-668c2f427280-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:13.790611 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.790460    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwzzg\" (UniqueName: \"kubernetes.io/projected/e1069ab2-ad70-48ec-8a8e-668c2f427280-kube-api-access-zwzzg\") pod \"payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:13.891692 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.891588    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1069ab2-ad70-48ec-8a8e-668c2f427280-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:13.891692 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.891689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwzzg\" (UniqueName: \"kubernetes.io/projected/e1069ab2-ad70-48ec-8a8e-668c2f427280-kube-api-access-zwzzg\") pod \"payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:13.892346 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.892324    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1069ab2-ad70-48ec-8a8e-668c2f427280-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:13.908368 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:13.908341    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwzzg\" (UniqueName: \"kubernetes.io/projected/e1069ab2-ad70-48ec-8a8e-668c2f427280-kube-api-access-zwzzg\") pod \"payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:14.043467 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.043401    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:14.326718 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.326691    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"]
Sep 10 21:10:14.329105 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:14.329078    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1069ab2_ad70_48ec_8a8e_668c2f427280.slice/crio-e29e76132f794d2ffe4d0654d179d33ffa8f8f8bd77198b34d869c4b2c9273d1 WatchSource:0}: Error finding container e29e76132f794d2ffe4d0654d179d33ffa8f8f8bd77198b34d869c4b2c9273d1: Status 404 returned error can't find the container with id e29e76132f794d2ffe4d0654d179d33ffa8f8f8bd77198b34d869c4b2c9273d1
Sep 10 21:10:14.401990 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.401959    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r" event={"ID":"e1069ab2-ad70-48ec-8a8e-668c2f427280","Type":"ContainerStarted","Data":"0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe"}
Sep 10 21:10:14.402128 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.402019    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r" event={"ID":"e1069ab2-ad70-48ec-8a8e-668c2f427280","Type":"ContainerStarted","Data":"e29e76132f794d2ffe4d0654d179d33ffa8f8f8bd77198b34d869c4b2c9273d1"}
Sep 10 21:10:14.402128 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.402110    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:14.425140 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.425092    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r" podStartSLOduration=1.42507389 podStartE2EDuration="1.42507389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:14.422515167 +0000 UTC m=+708.748276087" watchObservedRunningTime="2026-09-10 21:10:14.42507389 +0000 UTC m=+708.750834810"
Sep 10 21:10:14.785718 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.785682    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"]
Sep 10 21:10:14.789788 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.789764    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:14.793173 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.793043    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-f96a7a-serving-cert\""
Sep 10 21:10:14.802771 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.802655    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"]
Sep 10 21:10:14.836361 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.836316    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:14.901119 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.901080    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-metrics-certs\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:14.901119 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.901121    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2cx6\" (UniqueName: \"kubernetes.io/projected/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-kube-api-access-v2cx6\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:14.901353 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.901160    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-maas-api-tls\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:14.901353 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.901206    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-openshift-service-ca\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:14.919357 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:14.919321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:15.002672 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.002637    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-metrics-certs\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.002843 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.002678    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2cx6\" (UniqueName: \"kubernetes.io/projected/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-kube-api-access-v2cx6\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.002843 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.002716    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-maas-api-tls\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.002843 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.002775    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-openshift-service-ca\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.003488 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.003461    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-openshift-service-ca\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.005394 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.005369    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-metrics-certs\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.005473 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.005378    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-maas-api-tls\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.011878 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.011849    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2cx6\" (UniqueName: \"kubernetes.io/projected/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-kube-api-access-v2cx6\") pod \"maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.016022 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.015987    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:15.103268 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.103178    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:15.363780 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.363699    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:15.384480 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.384451    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"]
Sep 10 21:10:15.386652 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:15.386626    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20abb0f6_ed7b_4ec9_bd16_619782b7dd63.slice/crio-a0fa8452cbcb109657c676ad51f1286d63c2aac69a5690eec597206d0b3ccf9b WatchSource:0}: Error finding container a0fa8452cbcb109657c676ad51f1286d63c2aac69a5690eec597206d0b3ccf9b: Status 404 returned error can't find the container with id a0fa8452cbcb109657c676ad51f1286d63c2aac69a5690eec597206d0b3ccf9b
Sep 10 21:10:15.409957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.409926    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp" event={"ID":"20abb0f6-ed7b-4ec9-bd16-619782b7dd63","Type":"ContainerStarted","Data":"a0fa8452cbcb109657c676ad51f1286d63c2aac69a5690eec597206d0b3ccf9b"}
Sep 10 21:10:15.465105 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:15.465075    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:16.416977 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:16.416942    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp" event={"ID":"20abb0f6-ed7b-4ec9-bd16-619782b7dd63","Type":"ContainerStarted","Data":"510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5"}
Sep 10 21:10:16.417468 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:16.417001    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:16.434976 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:16.434916    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp" podStartSLOduration=2.02039832 podStartE2EDuration="2.43489834s" totalImagesPullingTime="414.50002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:16.43422469 +0000 UTC m=+710.759985614" watchObservedRunningTime="2026-09-10 21:10:16.43489834 +0000 UTC m=+710.760659259"
Sep 10 21:10:17.486616 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:17.486576    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:17.621393 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:17.621357    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:17.720658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:17.720621    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:17.827516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:17.827403    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:18.367862 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:18.367835    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:10:18.368058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:18.367901    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:10:20.485512 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:20.485478    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:22.047186 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:22.047148    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:22.427758 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:22.427729    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:10:22.730872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:22.730802    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:22.730872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:22.730847    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:22.743942 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:22.743902    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:23.462388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:23.462353    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:10:25.412233 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:25.412198    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:26.243350 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:26.243317    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:28.945522 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:28.945487    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:29.246311 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:29.246219    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:29.560170 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:29.560076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:29.697368 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:29.697333    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:32.227066 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:32.227028    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:32.320277 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:32.320247    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:33.345797 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.345766    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"]
Sep 10 21:10:33.351430 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.351395    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.353932 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.353909    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-31e7cb-serving-cert\""
Sep 10 21:10:33.362786 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.362763    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"]
Sep 10 21:10:33.393186 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.393154    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:33.458758 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.458726    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:33.470330 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.470301    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c31fd1f-8234-427a-a1b0-8c864cf1497a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.470516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.470343    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.470516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.470374    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-metrics-certs\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.470516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.470487    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch7pr\" (UniqueName: \"kubernetes.io/projected/9c31fd1f-8234-427a-a1b0-8c864cf1497a-kube-api-access-ch7pr\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.571312 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.571282    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c31fd1f-8234-427a-a1b0-8c864cf1497a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.571533 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.571322    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.571533 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.571442    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-metrics-certs\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.571533 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.571501    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch7pr\" (UniqueName: \"kubernetes.io/projected/9c31fd1f-8234-427a-a1b0-8c864cf1497a-kube-api-access-ch7pr\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.572078 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:10:33.572045    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-serving-cert: secret "maas-api-e2e-inf-b-31e7cb-serving-cert" not found
Sep 10 21:10:33.572190 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:10:33.572129    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls podName:9c31fd1f-8234-427a-a1b0-8c864cf1497a nodeName:}" failed. No retries permitted until 2026-09-10 21:10:34.072103843 +0000 UTC m=+728.397864742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls") pod "maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k" (UID: "9c31fd1f-8234-427a-a1b0-8c864cf1497a") : secret "maas-api-e2e-inf-b-31e7cb-serving-cert" not found
Sep 10 21:10:33.572263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.572215    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c31fd1f-8234-427a-a1b0-8c864cf1497a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.574195 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.574176    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-metrics-certs\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.584163 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.584141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch7pr\" (UniqueName: \"kubernetes.io/projected/9c31fd1f-8234-427a-a1b0-8c864cf1497a-kube-api-access-ch7pr\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:33.717246 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:33.717209    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:34.010492 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.010395    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:34.078046 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.078012    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:34.080432 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.080396    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls\") pod \"maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:34.108159 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.108119    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:34.186597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.186561    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"]
Sep 10 21:10:34.190573 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.190548    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.193046 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.193024    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-31e7cb\""
Sep 10 21:10:34.193184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.193136    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-31e7cb-dockercfg-wsf29\""
Sep 10 21:10:34.199507 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.199478    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"]
Sep 10 21:10:34.265344 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.265269    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:34.279772 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.279738    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd17a102-039d-41c9-93b6-6ff691112ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.279909 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.279884    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbl58\" (UniqueName: \"kubernetes.io/projected/bd17a102-039d-41c9-93b6-6ff691112ac4-kube-api-access-lbl58\") pod \"payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.381200 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.381156    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd17a102-039d-41c9-93b6-6ff691112ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.381794 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.381770    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbl58\" (UniqueName: \"kubernetes.io/projected/bd17a102-039d-41c9-93b6-6ff691112ac4-kube-api-access-lbl58\") pod \"payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.382125 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.382095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd17a102-039d-41c9-93b6-6ff691112ac4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.392590 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.392555    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbl58\" (UniqueName: \"kubernetes.io/projected/bd17a102-039d-41c9-93b6-6ff691112ac4-kube-api-access-lbl58\") pod \"payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.502267 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.502236    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:34.546362 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.546327    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"]
Sep 10 21:10:34.547031 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:34.546999    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c31fd1f_8234_427a_a1b0_8c864cf1497a.slice/crio-8e105d6dc5a65d8aa0467b2eb2cb983aa672fd14209d55acabac4c4d4d5510c5 WatchSource:0}: Error finding container 8e105d6dc5a65d8aa0467b2eb2cb983aa672fd14209d55acabac4c4d4d5510c5: Status 404 returned error can't find the container with id 8e105d6dc5a65d8aa0467b2eb2cb983aa672fd14209d55acabac4c4d4d5510c5
Sep 10 21:10:34.586278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.586245    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:34.783291 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:34.783261    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"]
Sep 10 21:10:34.787079 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:10:34.787055    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd17a102_039d_41c9_93b6_6ff691112ac4.slice/crio-d5c1c9e15512ab01e9b02aecf60d79dd6d9b377d68c3481db4ce3e8c3d704f89 WatchSource:0}: Error finding container d5c1c9e15512ab01e9b02aecf60d79dd6d9b377d68c3481db4ce3e8c3d704f89: Status 404 returned error can't find the container with id d5c1c9e15512ab01e9b02aecf60d79dd6d9b377d68c3481db4ce3e8c3d704f89
Sep 10 21:10:35.466163 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.466122    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:35.513365 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.513325    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k" event={"ID":"9c31fd1f-8234-427a-a1b0-8c864cf1497a","Type":"ContainerStarted","Data":"b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155"}
Sep 10 21:10:35.513573 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.513390    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k" event={"ID":"9c31fd1f-8234-427a-a1b0-8c864cf1497a","Type":"ContainerStarted","Data":"8e105d6dc5a65d8aa0467b2eb2cb983aa672fd14209d55acabac4c4d4d5510c5"}
Sep 10 21:10:35.513573 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.513434    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:35.515030 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.515006    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt" event={"ID":"bd17a102-039d-41c9-93b6-6ff691112ac4","Type":"ContainerStarted","Data":"8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3"}
Sep 10 21:10:35.515144 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.515046    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt" event={"ID":"bd17a102-039d-41c9-93b6-6ff691112ac4","Type":"ContainerStarted","Data":"d5c1c9e15512ab01e9b02aecf60d79dd6d9b377d68c3481db4ce3e8c3d704f89"}
Sep 10 21:10:35.515144 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.515135    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:35.544138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.544088    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k" podStartSLOduration=2.191491107 podStartE2EDuration="2.54407346s" totalImagesPullingTime="352.582353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:35.542364533 +0000 UTC m=+729.868125453" watchObservedRunningTime="2026-09-10 21:10:35.54407346 +0000 UTC m=+729.869834380"
Sep 10 21:10:35.568247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:35.568200    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt" podStartSLOduration=1.568183587 podStartE2EDuration="1.568183587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:35.566240123 +0000 UTC m=+729.892001042" watchObservedRunningTime="2026-09-10 21:10:35.568183587 +0000 UTC m=+729.893944508"
Sep 10 21:10:36.272807 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:36.272756    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:37.192757 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:37.192709    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:38.143868 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:38.143829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:39.169489 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:39.169457    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:39.619856 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:39.619774    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:39.733132 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:39.733092    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:39.884596 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:39.884515    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:41.525339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:41.525311    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:10:43.130384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:43.130341    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:45.995373 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:45.995335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:46.097711 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:46.097669    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:46.195385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:46.195345    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:46.319213 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:46.319117    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:46.522269 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:46.522241    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:10:53.069570 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.069537    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:53.314320 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.314291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:53.418579 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.418540    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:53.782853 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.782773    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:53.817388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.817351    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"]
Sep 10 21:10:53.817651 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.817623    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r" podUID="e1069ab2-ad70-48ec-8a8e-668c2f427280" containerName="payload-pre-processing" containerID="cri-o://0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe" gracePeriod=30
Sep 10 21:10:53.889296 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.889250    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:53.995913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:53.995873    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:54.082743 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.082714    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:54.091340 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.091309    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:54.169737 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.169704    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1069ab2-ad70-48ec-8a8e-668c2f427280-plugins-config-volume\") pod \"e1069ab2-ad70-48ec-8a8e-668c2f427280\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") "
Sep 10 21:10:54.169928 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.169795    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwzzg\" (UniqueName: \"kubernetes.io/projected/e1069ab2-ad70-48ec-8a8e-668c2f427280-kube-api-access-zwzzg\") pod \"e1069ab2-ad70-48ec-8a8e-668c2f427280\" (UID: \"e1069ab2-ad70-48ec-8a8e-668c2f427280\") "
Sep 10 21:10:54.170176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.170132    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1069ab2-ad70-48ec-8a8e-668c2f427280-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e1069ab2-ad70-48ec-8a8e-668c2f427280" (UID: "e1069ab2-ad70-48ec-8a8e-668c2f427280"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:10:54.175069 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.174997    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1069ab2-ad70-48ec-8a8e-668c2f427280-kube-api-access-zwzzg" (OuterVolumeSpecName: "kube-api-access-zwzzg") pod "e1069ab2-ad70-48ec-8a8e-668c2f427280" (UID: "e1069ab2-ad70-48ec-8a8e-668c2f427280"). InnerVolumeSpecName "kube-api-access-zwzzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:10:54.217323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.217293    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:54.271527 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.271484    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1069ab2-ad70-48ec-8a8e-668c2f427280-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.271527 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.271526    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwzzg\" (UniqueName: \"kubernetes.io/projected/e1069ab2-ad70-48ec-8a8e-668c2f427280-kube-api-access-zwzzg\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.579868 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.579774    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:54.618998 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.618963    2584 generic.go:356] "Generic (PLEG): container finished" podID="e1069ab2-ad70-48ec-8a8e-668c2f427280" containerID="0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe" exitCode=0
Sep 10 21:10:54.619180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.619005    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"
Sep 10 21:10:54.619180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.619041    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r" event={"ID":"e1069ab2-ad70-48ec-8a8e-668c2f427280","Type":"ContainerDied","Data":"0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe"}
Sep 10 21:10:54.619180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.619100    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r" event={"ID":"e1069ab2-ad70-48ec-8a8e-668c2f427280","Type":"ContainerDied","Data":"e29e76132f794d2ffe4d0654d179d33ffa8f8f8bd77198b34d869c4b2c9273d1"}
Sep 10 21:10:54.619180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.619123    2584 scope.go:117] "RemoveContainer" containerID="0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe"
Sep 10 21:10:54.628066 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.628043    2584 scope.go:117] "RemoveContainer" containerID="0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe"
Sep 10 21:10:54.628378 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:10:54.628356    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe\": container with ID starting with 0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe not found: ID does not exist" containerID="0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe"
Sep 10 21:10:54.628486 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.628390    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe"} err="failed to get container status \"0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe\": rpc error: code = NotFound desc = could not find container \"0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe\": container with ID starting with 0cb5ce78f5202e89eaa3840568a6c76e7b2b9a745690f485087471eb84c010fe not found: ID does not exist"
Sep 10 21:10:54.647435 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.647395    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"]
Sep 10 21:10:54.657271 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.657241    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d9fd5e6e-79f7d56ddb-t2h4r"]
Sep 10 21:10:54.683053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.683017    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:54.783215 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.783174    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:54.888286 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:54.888188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:55.193908 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:55.193871    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:55.382153 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:55.382107    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:56.301111 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:56.301074    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1069ab2-ad70-48ec-8a8e-668c2f427280" path="/var/lib/kubelet/pods/e1069ab2-ad70-48ec-8a8e-668c2f427280/volumes"
Sep 10 21:10:57.135472 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:57.135430    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:58.049465 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:58.049401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:58.154392 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:58.154352    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:58.255323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:58.255281    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:58.353765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:58.353685    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:58.618210 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:58.618116    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:59.066211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:59.066176    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:59.200383 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:10:59.200347    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:29.973707 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:29.973669    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:30.075086 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:30.075049    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:30.198241 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:30.198203    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:30.302220 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:30.302136    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:30.413298 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:30.413266    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:30.989080 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:30.989023    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:31.258710 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:31.258626    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:31.386927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:31.386890    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:34.098799 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:34.098765    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:34.262884 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:34.262850    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:35.475455 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:35.475397    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:35.677860 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:35.677820    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:35.817798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:35.817710    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.312597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.312559    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.414495 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.414458    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.487693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.487657    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"]
Sep 10 21:11:36.488231 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.488213    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1069ab2-ad70-48ec-8a8e-668c2f427280" containerName="payload-pre-processing"
Sep 10 21:11:36.488231 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.488234    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1069ab2-ad70-48ec-8a8e-668c2f427280" containerName="payload-pre-processing"
Sep 10 21:11:36.488322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.488307    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1069ab2-ad70-48ec-8a8e-668c2f427280" containerName="payload-pre-processing"
Sep 10 21:11:36.492837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.492820    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:36.495137 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.495114    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e95f46e9-dockercfg-9t289\""
Sep 10 21:11:36.495773 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.495754    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e95f46e9\""
Sep 10 21:11:36.500547 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.500521    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"]
Sep 10 21:11:36.577673 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.577577    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7430a377-645a-40d6-b62a-c0462935ec2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:36.577826 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.577717    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gz8t\" (UniqueName: \"kubernetes.io/projected/7430a377-645a-40d6-b62a-c0462935ec2e-kube-api-access-6gz8t\") pod \"payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:36.678636 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.678598    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gz8t\" (UniqueName: \"kubernetes.io/projected/7430a377-645a-40d6-b62a-c0462935ec2e-kube-api-access-6gz8t\") pod \"payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:36.678831 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.678696    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7430a377-645a-40d6-b62a-c0462935ec2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:36.679325 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.679305    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7430a377-645a-40d6-b62a-c0462935ec2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:36.687364 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.687332    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gz8t\" (UniqueName: \"kubernetes.io/projected/7430a377-645a-40d6-b62a-c0462935ec2e-kube-api-access-6gz8t\") pod \"payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:36.765957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.765917    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.805978 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:36.805943    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:37.082256 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:37.082227    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"]
Sep 10 21:11:37.084655 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:11:37.084616    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7430a377_645a_40d6_b62a_c0462935ec2e.slice/crio-620c32610e4460f0c0b4374fe88cbad47ce9c21b2a8e70e83500a59a87c47623 WatchSource:0}: Error finding container 620c32610e4460f0c0b4374fe88cbad47ce9c21b2a8e70e83500a59a87c47623: Status 404 returned error can't find the container with id 620c32610e4460f0c0b4374fe88cbad47ce9c21b2a8e70e83500a59a87c47623
Sep 10 21:11:37.846822 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:37.846785    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv" event={"ID":"7430a377-645a-40d6-b62a-c0462935ec2e","Type":"ContainerStarted","Data":"e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c"}
Sep 10 21:11:37.847224 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:37.846868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv" event={"ID":"7430a377-645a-40d6-b62a-c0462935ec2e","Type":"ContainerStarted","Data":"620c32610e4460f0c0b4374fe88cbad47ce9c21b2a8e70e83500a59a87c47623"}
Sep 10 21:11:37.847224 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:37.846928    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:48.854198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:48.854119    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:11:48.873968 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:48.873901    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv" podStartSLOduration=12.873880453 podStartE2EDuration="12.873880453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:37.868493202 +0000 UTC m=+792.194254121" watchObservedRunningTime="2026-09-10 21:11:48.873880453 +0000 UTC m=+803.199641373"
Sep 10 21:11:50.722172 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.722137    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"]
Sep 10 21:11:50.725933 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.725911    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.728392 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.728362    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-75ed5f65-kserve-self-signed-certs\""
Sep 10 21:11:50.740054 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.740022    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"]
Sep 10 21:11:50.762593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.762535    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:50.809301 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.809259    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-model-cache\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.809628 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.809601    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tmp-dir\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.809798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.809781    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tls-certs\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.809926 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.809910    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-home\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.810071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.810042    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m7ps\" (UniqueName: \"kubernetes.io/projected/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-kube-api-access-4m7ps\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.810188 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.810106    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-dshm\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911332 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911296    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-model-cache\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911591 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911346    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tmp-dir\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911591 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911370    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tls-certs\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911591 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911395    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-home\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911591 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911467    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m7ps\" (UniqueName: \"kubernetes.io/projected/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-kube-api-access-4m7ps\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911591 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911511    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-dshm\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911880 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911768    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-model-cache\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911880 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911816    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tmp-dir\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.911880 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.911863    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-home\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.914239 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.914217    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-dshm\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.914942 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.914918    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tls-certs\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:50.920307 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:50.920279    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m7ps\" (UniqueName: \"kubernetes.io/projected/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-kube-api-access-4m7ps\") pod \"e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") " pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:51.036247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:51.036162    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:11:51.308413 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:51.308385    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"]
Sep 10 21:11:51.310669 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:11:51.310643    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10bc4d11_f7c8_4b30_9599_dc28ac5d7953.slice/crio-84201f747ca21a32c3aa82b068d7e278238a5aad0fbcb04bcdcd75ddc2aa7135 WatchSource:0}: Error finding container 84201f747ca21a32c3aa82b068d7e278238a5aad0fbcb04bcdcd75ddc2aa7135: Status 404 returned error can't find the container with id 84201f747ca21a32c3aa82b068d7e278238a5aad0fbcb04bcdcd75ddc2aa7135
Sep 10 21:11:51.922524 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:51.922485    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm" event={"ID":"10bc4d11-f7c8-4b30-9599-dc28ac5d7953","Type":"ContainerStarted","Data":"14103acad3cd52fa3b7ff85d30c2edcc0c6e58a65d9bdc327443336e3402c993"}
Sep 10 21:11:51.922942 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:51.922539    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm" event={"ID":"10bc4d11-f7c8-4b30-9599-dc28ac5d7953","Type":"ContainerStarted","Data":"84201f747ca21a32c3aa82b068d7e278238a5aad0fbcb04bcdcd75ddc2aa7135"}
Sep 10 21:11:51.940793 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:11:51.940729    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm" podStartSLOduration=1.940713615 podStartE2EDuration="1.940713615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:51.939134892 +0000 UTC m=+806.264895831" watchObservedRunningTime="2026-09-10 21:11:51.940713615 +0000 UTC m=+806.266474536"
Sep 10 21:12:01.036949 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:01.036908    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:12:01.037586 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:01.036971    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:12:01.049368 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:01.049343    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:12:01.985317 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:01.985287    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:12:06.197982 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.197944    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.232026 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.231972    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"]
Sep 10 21:12:06.232784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.232723    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv" podUID="7430a377-645a-40d6-b62a-c0462935ec2e" containerName="payload-pre-processing" containerID="cri-o://e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c" gracePeriod=30
Sep 10 21:12:06.298476 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.298436    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.396288 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.396248    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.465703 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.465610    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.506445 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.506393    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:12:06.665732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.665689    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7430a377-645a-40d6-b62a-c0462935ec2e-plugins-config-volume\") pod \"7430a377-645a-40d6-b62a-c0462935ec2e\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") "
Sep 10 21:12:06.665909 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.665805    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gz8t\" (UniqueName: \"kubernetes.io/projected/7430a377-645a-40d6-b62a-c0462935ec2e-kube-api-access-6gz8t\") pod \"7430a377-645a-40d6-b62a-c0462935ec2e\" (UID: \"7430a377-645a-40d6-b62a-c0462935ec2e\") "
Sep 10 21:12:06.666089 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.666063    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7430a377-645a-40d6-b62a-c0462935ec2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7430a377-645a-40d6-b62a-c0462935ec2e" (UID: "7430a377-645a-40d6-b62a-c0462935ec2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:06.667933 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.667906    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7430a377-645a-40d6-b62a-c0462935ec2e-kube-api-access-6gz8t" (OuterVolumeSpecName: "kube-api-access-6gz8t") pod "7430a377-645a-40d6-b62a-c0462935ec2e" (UID: "7430a377-645a-40d6-b62a-c0462935ec2e"). InnerVolumeSpecName "kube-api-access-6gz8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:06.766800 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.766717    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7430a377-645a-40d6-b62a-c0462935ec2e-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:06.766800 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:06.766745    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gz8t\" (UniqueName: \"kubernetes.io/projected/7430a377-645a-40d6-b62a-c0462935ec2e-kube-api-access-6gz8t\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:07.004233 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.004192    2584 generic.go:356] "Generic (PLEG): container finished" podID="7430a377-645a-40d6-b62a-c0462935ec2e" containerID="e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c" exitCode=0
Sep 10 21:12:07.004445 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.004236    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv" event={"ID":"7430a377-645a-40d6-b62a-c0462935ec2e","Type":"ContainerDied","Data":"e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c"}
Sep 10 21:12:07.004445 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.004318    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv" event={"ID":"7430a377-645a-40d6-b62a-c0462935ec2e","Type":"ContainerDied","Data":"620c32610e4460f0c0b4374fe88cbad47ce9c21b2a8e70e83500a59a87c47623"}
Sep 10 21:12:07.004445 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.004250    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"
Sep 10 21:12:07.004445 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.004352    2584 scope.go:117] "RemoveContainer" containerID="e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c"
Sep 10 21:12:07.013547 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.013524    2584 scope.go:117] "RemoveContainer" containerID="e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c"
Sep 10 21:12:07.013815 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:12:07.013789    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c\": container with ID starting with e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c not found: ID does not exist" containerID="e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c"
Sep 10 21:12:07.013883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.013823    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c"} err="failed to get container status \"e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c\": rpc error: code = NotFound desc = could not find container \"e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c\": container with ID starting with e126b378c5c57618379e10e897f965ed4f9563035284c912962b13cf81e9bf2c not found: ID does not exist"
Sep 10 21:12:07.037220 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.037163    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"]
Sep 10 21:12:07.051712 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.051689    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e95f46e9-58cf866c54-57jzv"]
Sep 10 21:12:07.583201 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:07.583169    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:08.001112 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:08.001072    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:08.283523 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:08.283407    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:08.304029 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:08.304001    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7430a377-645a-40d6-b62a-c0462935ec2e" path="/var/lib/kubelet/pods/7430a377-645a-40d6-b62a-c0462935ec2e/volumes"
Sep 10 21:12:08.546376 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:08.546298    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:11.319610 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:11.319575    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:11.420399 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:11.420363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:12.637739 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:12.637700    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:12.808075 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:12.808036    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:13.010381 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:13.010340    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:13.195682 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:13.195641    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:13.289602 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:13.289512    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:18.837516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:18.837482    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:25.533767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.533718    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"]
Sep 10 21:12:25.534211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.534125    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7430a377-645a-40d6-b62a-c0462935ec2e" containerName="payload-pre-processing"
Sep 10 21:12:25.534211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.534136    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7430a377-645a-40d6-b62a-c0462935ec2e" containerName="payload-pre-processing"
Sep 10 21:12:25.534211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.534211    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7430a377-645a-40d6-b62a-c0462935ec2e" containerName="payload-pre-processing"
Sep 10 21:12:25.537298 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.537282    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.540512 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.540490    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-76b274\"/\"default-dockercfg-hr5b6\""
Sep 10 21:12:25.540632 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.540539    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-76b274\"/\"kube-root-ca.crt\""
Sep 10 21:12:25.540697 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.540647    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-76b274\"/\"openshift-service-ca.crt\""
Sep 10 21:12:25.540697 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.540667    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-76b274\"/\"test-model-76b274-kserve-self-signed-certs\""
Sep 10 21:12:25.548119 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.548094    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"]
Sep 10 21:12:25.650783 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.650751    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tls-certs\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.650981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.650789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5slw\" (UniqueName: \"kubernetes.io/projected/e5a6b291-05eb-4ab0-ae13-57206bb057f5-kube-api-access-r5slw\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.650981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.650870    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-home\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.650981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.650936    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-dshm\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.650981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.650961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-model-cache\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.651202 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.650984    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tmp-dir\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.651321 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.651292    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:25.751993 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.751951    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-home\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752008    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-dshm\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752034    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-model-cache\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752053    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tmp-dir\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752174    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tls-certs\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752199    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5slw\" (UniqueName: \"kubernetes.io/projected/e5a6b291-05eb-4ab0-ae13-57206bb057f5-kube-api-access-r5slw\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752374    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-home\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752538 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752467    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-model-cache\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.752538 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.752515    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tmp-dir\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.754450 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.754399    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-dshm\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.754611 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.754595    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tls-certs\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.763876 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.763844    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5slw\" (UniqueName: \"kubernetes.io/projected/e5a6b291-05eb-4ab0-ae13-57206bb057f5-kube-api-access-r5slw\") pod \"test-model-76b274-kserve-69d45fcb89-l7lwf\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") " pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:25.846901 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:25.846808    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:26.132131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:26.132102    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"]
Sep 10 21:12:26.133541 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:12:26.133514    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5a6b291_05eb_4ab0_ae13_57206bb057f5.slice/crio-69233ed8c57b6a27ac4d3bbef17ca075e4f029c7f4b8d7b6d1ac40b6cbffb0d9 WatchSource:0}: Error finding container 69233ed8c57b6a27ac4d3bbef17ca075e4f029c7f4b8d7b6d1ac40b6cbffb0d9: Status 404 returned error can't find the container with id 69233ed8c57b6a27ac4d3bbef17ca075e4f029c7f4b8d7b6d1ac40b6cbffb0d9
Sep 10 21:12:27.106514 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:27.106469    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf" event={"ID":"e5a6b291-05eb-4ab0-ae13-57206bb057f5","Type":"ContainerStarted","Data":"0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078"}
Sep 10 21:12:27.106910 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:27.106522    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf" event={"ID":"e5a6b291-05eb-4ab0-ae13-57206bb057f5","Type":"ContainerStarted","Data":"69233ed8c57b6a27ac4d3bbef17ca075e4f029c7f4b8d7b6d1ac40b6cbffb0d9"}
Sep 10 21:12:27.126846 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:27.126799    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf" podStartSLOduration=2.126783895 podStartE2EDuration="2.126783895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:27.125562484 +0000 UTC m=+841.451323405" watchObservedRunningTime="2026-09-10 21:12:27.126783895 +0000 UTC m=+841.452544816"
Sep 10 21:12:28.812386 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:28.812345    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"]
Sep 10 21:12:28.835863 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:28.835829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:29.909250 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:29.909212    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"]
Sep 10 21:12:29.938372 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:29.934365    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:31.372290 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:31.372242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:31.686808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:31.686764    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:33.781765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:33.781723    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:35.847784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:35.847750    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:35.848163 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:35.847803    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:35.860632 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:35.860602    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:36.160655 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:36.160622    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:12:41.630927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:41.630889    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:42.220090 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.220057    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"]
Sep 10 21:12:42.220306 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.220275    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp" podUID="20abb0f6-ed7b-4ec9-bd16-619782b7dd63" containerName="maas-api" containerID="cri-o://510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5" gracePeriod=30
Sep 10 21:12:42.255124 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.255088    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:42.330566 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.330527    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:42.393528 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.392960    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:42.471492 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.471349    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:42.479685 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.479658    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:12:42.533798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.533759    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:42.625647 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.625608    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-maas-api-tls\") pod \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") "
Sep 10 21:12:42.625837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.625688    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-openshift-service-ca\") pod \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") "
Sep 10 21:12:42.625837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.625710    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-metrics-certs\") pod \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") "
Sep 10 21:12:42.625837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.625777    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2cx6\" (UniqueName: \"kubernetes.io/projected/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-kube-api-access-v2cx6\") pod \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\" (UID: \"20abb0f6-ed7b-4ec9-bd16-619782b7dd63\") "
Sep 10 21:12:42.626109 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.626076    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20abb0f6-ed7b-4ec9-bd16-619782b7dd63" (UID: "20abb0f6-ed7b-4ec9-bd16-619782b7dd63"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:42.628052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.628019    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20abb0f6-ed7b-4ec9-bd16-619782b7dd63" (UID: "20abb0f6-ed7b-4ec9-bd16-619782b7dd63"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:42.628052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.628033    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "20abb0f6-ed7b-4ec9-bd16-619782b7dd63" (UID: "20abb0f6-ed7b-4ec9-bd16-619782b7dd63"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:42.628052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.628033    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-kube-api-access-v2cx6" (OuterVolumeSpecName: "kube-api-access-v2cx6") pod "20abb0f6-ed7b-4ec9-bd16-619782b7dd63" (UID: "20abb0f6-ed7b-4ec9-bd16-619782b7dd63"). InnerVolumeSpecName "kube-api-access-v2cx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:42.657084 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.657045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:42.727023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.726937    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.727023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.726965    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.727023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.726977    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.727023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.726986    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2cx6\" (UniqueName: \"kubernetes.io/projected/20abb0f6-ed7b-4ec9-bd16-619782b7dd63-kube-api-access-v2cx6\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.806379 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:42.806342    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.187604 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.187566    2584 generic.go:356] "Generic (PLEG): container finished" podID="20abb0f6-ed7b-4ec9-bd16-619782b7dd63" containerID="510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5" exitCode=0
Sep 10 21:12:43.187803 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.187618    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp" event={"ID":"20abb0f6-ed7b-4ec9-bd16-619782b7dd63","Type":"ContainerDied","Data":"510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5"}
Sep 10 21:12:43.187803 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.187622    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"
Sep 10 21:12:43.187803 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.187708    2584 scope.go:117] "RemoveContainer" containerID="510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5"
Sep 10 21:12:43.187958 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.187696    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp" event={"ID":"20abb0f6-ed7b-4ec9-bd16-619782b7dd63","Type":"ContainerDied","Data":"a0fa8452cbcb109657c676ad51f1286d63c2aac69a5690eec597206d0b3ccf9b"}
Sep 10 21:12:43.197333 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.197311    2584 scope.go:117] "RemoveContainer" containerID="510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5"
Sep 10 21:12:43.197656 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:12:43.197634    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5\": container with ID starting with 510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5 not found: ID does not exist" containerID="510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5"
Sep 10 21:12:43.197732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.197669    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5"} err="failed to get container status \"510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5\": rpc error: code = NotFound desc = could not find container \"510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5\": container with ID starting with 510db5145d7d98e4a0eee72075eaa76ef99bed2505e11dfc477ee4e6530a4fa5 not found: ID does not exist"
Sep 10 21:12:43.209710 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.209675    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"]
Sep 10 21:12:43.213404 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.213378    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp"]
Sep 10 21:12:43.351369 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.351330    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.429334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.429299    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.493498 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.493396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.561435 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.561382    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.701353 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.701315    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.832599 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:43.832504    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6" podUID="4b6e999e-873d-43a0-86e4-87289a59863c" containerName="main" containerID="cri-o://b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1" gracePeriod=45
Sep 10 21:12:44.095582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.095507    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:12:44.195198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.195161    2584 generic.go:356] "Generic (PLEG): container finished" podID="4b6e999e-873d-43a0-86e4-87289a59863c" containerID="b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1" exitCode=0
Sep 10 21:12:44.195198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.195202    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"
Sep 10 21:12:44.195436 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.195206    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6" event={"ID":"4b6e999e-873d-43a0-86e4-87289a59863c","Type":"ContainerDied","Data":"b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1"}
Sep 10 21:12:44.195436 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.195236    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6" event={"ID":"4b6e999e-873d-43a0-86e4-87289a59863c","Type":"ContainerDied","Data":"c443100c0c3b2d3d2b7225db819707033a6fafdf8512321d3ecfacbc65509532"}
Sep 10 21:12:44.195436 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.195253    2584 scope.go:117] "RemoveContainer" containerID="b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1"
Sep 10 21:12:44.203490 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.203469    2584 scope.go:117] "RemoveContainer" containerID="b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1"
Sep 10 21:12:44.203794 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:12:44.203776    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1\": container with ID starting with b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1 not found: ID does not exist" containerID="b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1"
Sep 10 21:12:44.203844 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.203804    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1"} err="failed to get container status \"b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1\": rpc error: code = NotFound desc = could not find container \"b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1\": container with ID starting with b624b7c48169df0b8866bfb904755da54a4db18cbd5f8614adc676cee061fbd1 not found: ID does not exist"
Sep 10 21:12:44.242547 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242517    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-dshm\") pod \"4b6e999e-873d-43a0-86e4-87289a59863c\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") "
Sep 10 21:12:44.242674 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242572    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-model-cache\") pod \"4b6e999e-873d-43a0-86e4-87289a59863c\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") "
Sep 10 21:12:44.242674 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242640    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-home\") pod \"4b6e999e-873d-43a0-86e4-87289a59863c\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") "
Sep 10 21:12:44.242874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242691    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqpxz\" (UniqueName: \"kubernetes.io/projected/4b6e999e-873d-43a0-86e4-87289a59863c-kube-api-access-lqpxz\") pod \"4b6e999e-873d-43a0-86e4-87289a59863c\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") "
Sep 10 21:12:44.242874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242741    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-tmp-dir\") pod \"4b6e999e-873d-43a0-86e4-87289a59863c\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") "
Sep 10 21:12:44.242874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242770    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b6e999e-873d-43a0-86e4-87289a59863c-tls-certs\") pod \"4b6e999e-873d-43a0-86e4-87289a59863c\" (UID: \"4b6e999e-873d-43a0-86e4-87289a59863c\") "
Sep 10 21:12:44.243057 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242863    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b6e999e-873d-43a0-86e4-87289a59863c" (UID: "4b6e999e-873d-43a0-86e4-87289a59863c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:44.243057 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242948    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-home" (OuterVolumeSpecName: "home") pod "4b6e999e-873d-43a0-86e4-87289a59863c" (UID: "4b6e999e-873d-43a0-86e4-87289a59863c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:44.243057 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.242986    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b6e999e-873d-43a0-86e4-87289a59863c" (UID: "4b6e999e-873d-43a0-86e4-87289a59863c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:44.243204 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.243141    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-home\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.243204 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.243160    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-tmp-dir\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.243204 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.243177    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-model-cache\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.244826 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.244799    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b6e999e-873d-43a0-86e4-87289a59863c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b6e999e-873d-43a0-86e4-87289a59863c" (UID: "4b6e999e-873d-43a0-86e4-87289a59863c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:44.244918 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.244851    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6e999e-873d-43a0-86e4-87289a59863c-kube-api-access-lqpxz" (OuterVolumeSpecName: "kube-api-access-lqpxz") pod "4b6e999e-873d-43a0-86e4-87289a59863c" (UID: "4b6e999e-873d-43a0-86e4-87289a59863c"). InnerVolumeSpecName "kube-api-access-lqpxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:44.244918 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.244886    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-dshm" (OuterVolumeSpecName: "dshm") pod "4b6e999e-873d-43a0-86e4-87289a59863c" (UID: "4b6e999e-873d-43a0-86e4-87289a59863c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:44.301594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.301551    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20abb0f6-ed7b-4ec9-bd16-619782b7dd63" path="/var/lib/kubelet/pods/20abb0f6-ed7b-4ec9-bd16-619782b7dd63/volumes"
Sep 10 21:12:44.343881 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.343847    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b6e999e-873d-43a0-86e4-87289a59863c-dshm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.343881 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.343878    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqpxz\" (UniqueName: \"kubernetes.io/projected/4b6e999e-873d-43a0-86e4-87289a59863c-kube-api-access-lqpxz\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.344123 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.343895    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b6e999e-873d-43a0-86e4-87289a59863c-tls-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.344810 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.344778    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.452098 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.452065    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.516451 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.516402    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"]
Sep 10 21:12:44.519040 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.519016    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-75ed5f65-kserve-5b5ff77b-8h9p6"]
Sep 10 21:12:44.541094 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.541062    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.946338 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:44.946286    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm" podUID="10bc4d11-f7c8-4b30-9599-dc28ac5d7953" containerName="main" containerID="cri-o://14103acad3cd52fa3b7ff85d30c2edcc0c6e58a65d9bdc327443336e3402c993" gracePeriod=45
Sep 10 21:12:45.204535 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.204451    2584 generic.go:356] "Generic (PLEG): container finished" podID="10bc4d11-f7c8-4b30-9599-dc28ac5d7953" containerID="14103acad3cd52fa3b7ff85d30c2edcc0c6e58a65d9bdc327443336e3402c993" exitCode=0
Sep 10 21:12:45.204535 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.204505    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm" event={"ID":"10bc4d11-f7c8-4b30-9599-dc28ac5d7953","Type":"ContainerDied","Data":"14103acad3cd52fa3b7ff85d30c2edcc0c6e58a65d9bdc327443336e3402c993"}
Sep 10 21:12:45.204741 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.204542    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm" event={"ID":"10bc4d11-f7c8-4b30-9599-dc28ac5d7953","Type":"ContainerDied","Data":"84201f747ca21a32c3aa82b068d7e278238a5aad0fbcb04bcdcd75ddc2aa7135"}
Sep 10 21:12:45.204741 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.204558    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84201f747ca21a32c3aa82b068d7e278238a5aad0fbcb04bcdcd75ddc2aa7135"
Sep 10 21:12:45.212927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.212902    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:12:45.250721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.250688    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-home\") pod \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") "
Sep 10 21:12:45.250721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.250724    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-dshm\") pod \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") "
Sep 10 21:12:45.250955 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.250764    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tls-certs\") pod \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") "
Sep 10 21:12:45.250955 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.250788    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4m7ps\" (UniqueName: \"kubernetes.io/projected/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-kube-api-access-4m7ps\") pod \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") "
Sep 10 21:12:45.250955 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.250887    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tmp-dir\") pod \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") "
Sep 10 21:12:45.251119 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.250954    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-model-cache\") pod \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\" (UID: \"10bc4d11-f7c8-4b30-9599-dc28ac5d7953\") "
Sep 10 21:12:45.251119 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.250997    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-home" (OuterVolumeSpecName: "home") pod "10bc4d11-f7c8-4b30-9599-dc28ac5d7953" (UID: "10bc4d11-f7c8-4b30-9599-dc28ac5d7953"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:45.251232 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.251187    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "10bc4d11-f7c8-4b30-9599-dc28ac5d7953" (UID: "10bc4d11-f7c8-4b30-9599-dc28ac5d7953"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:45.251326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.251309    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-home\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:45.251326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.251309    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-model-cache" (OuterVolumeSpecName: "model-cache") pod "10bc4d11-f7c8-4b30-9599-dc28ac5d7953" (UID: "10bc4d11-f7c8-4b30-9599-dc28ac5d7953"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:45.251435 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.251326    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tmp-dir\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:45.253034 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.253002    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-dshm" (OuterVolumeSpecName: "dshm") pod "10bc4d11-f7c8-4b30-9599-dc28ac5d7953" (UID: "10bc4d11-f7c8-4b30-9599-dc28ac5d7953"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:12:45.253144 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.253070    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-kube-api-access-4m7ps" (OuterVolumeSpecName: "kube-api-access-4m7ps") pod "10bc4d11-f7c8-4b30-9599-dc28ac5d7953" (UID: "10bc4d11-f7c8-4b30-9599-dc28ac5d7953"). InnerVolumeSpecName "kube-api-access-4m7ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:45.253322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.253303    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "10bc4d11-f7c8-4b30-9599-dc28ac5d7953" (UID: "10bc4d11-f7c8-4b30-9599-dc28ac5d7953"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:45.351917 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.351871    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-model-cache\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:45.351917 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.351907    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-dshm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:45.351917 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.351920    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-tls-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:45.351917 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.351929    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m7ps\" (UniqueName: \"kubernetes.io/projected/10bc4d11-f7c8-4b30-9599-dc28ac5d7953-kube-api-access-4m7ps\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:12:45.424157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.424109    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f96a7a-79ddbb768c-vndlp" podUID="20abb0f6-ed7b-4ec9-bd16-619782b7dd63" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.45:8443/health\": context deadline exceeded"
Sep 10 21:12:45.904273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:45.904228    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:46.210486 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:46.210376    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"
Sep 10 21:12:46.231809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:46.231775    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"]
Sep 10 21:12:46.236071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:46.236046    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-75ed5f65-kserve-76697b6f54-qvhhm"]
Sep 10 21:12:46.302056 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:46.302019    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10bc4d11-f7c8-4b30-9599-dc28ac5d7953" path="/var/lib/kubelet/pods/10bc4d11-f7c8-4b30-9599-dc28ac5d7953/volumes"
Sep 10 21:12:46.302438 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:46.302408    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b6e999e-873d-43a0-86e4-87289a59863c" path="/var/lib/kubelet/pods/4b6e999e-873d-43a0-86e4-87289a59863c/volumes"
Sep 10 21:12:48.803947 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:48.803901    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.913115 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:48.913076    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:49.152118 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:49.152089    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:49.534917 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:49.534827    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:49.902988 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:49.902956    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:50.520458 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:50.520402    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:52.417890 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:52.417856    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:52.519580 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:52.519546    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:52.618445 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:52.618393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:52.718651 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:52.718553    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:53.054388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:53.054296    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:53.147769 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:53.147726    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:54.369552 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:54.369506    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:54.436771 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:54.436730    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:54.536593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:54.536558    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:54.892090 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:54.892058    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:54.992694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:54.992659    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:55.016561 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.016528    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"]
Sep 10 21:12:55.016927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.016916    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6e999e-873d-43a0-86e4-87289a59863c" containerName="main"
Sep 10 21:12:55.016965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.016929    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6e999e-873d-43a0-86e4-87289a59863c" containerName="main"
Sep 10 21:12:55.016965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.016939    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20abb0f6-ed7b-4ec9-bd16-619782b7dd63" containerName="maas-api"
Sep 10 21:12:55.016965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.016945    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="20abb0f6-ed7b-4ec9-bd16-619782b7dd63" containerName="maas-api"
Sep 10 21:12:55.016965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.016958    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10bc4d11-f7c8-4b30-9599-dc28ac5d7953" containerName="main"
Sep 10 21:12:55.016965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.016964    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="10bc4d11-f7c8-4b30-9599-dc28ac5d7953" containerName="main"
Sep 10 21:12:55.017134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.017020    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="10bc4d11-f7c8-4b30-9599-dc28ac5d7953" containerName="main"
Sep 10 21:12:55.017134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.017031    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6e999e-873d-43a0-86e4-87289a59863c" containerName="main"
Sep 10 21:12:55.017134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.017037    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="20abb0f6-ed7b-4ec9-bd16-619782b7dd63" containerName="maas-api"
Sep 10 21:12:55.019916 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.019899    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.022254 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.022235    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-ea71f9\""
Sep 10 21:12:55.022484 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.022464    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-ea71f9-dockercfg-sknpm\""
Sep 10 21:12:55.035018 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.034994    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"]
Sep 10 21:12:55.041003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.040979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f7df\" (UniqueName: \"kubernetes.io/projected/95b9c1f4-445a-48e1-ad63-24fd1e329233-kube-api-access-6f7df\") pod \"payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") " pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.041120 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.041009    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c1f4-445a-48e1-ad63-24fd1e329233-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") " pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.142222 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.142131    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f7df\" (UniqueName: \"kubernetes.io/projected/95b9c1f4-445a-48e1-ad63-24fd1e329233-kube-api-access-6f7df\") pod \"payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") " pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.142222 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.142186    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c1f4-445a-48e1-ad63-24fd1e329233-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") " pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.143023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.143002    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c1f4-445a-48e1-ad63-24fd1e329233-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") " pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.154238 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.154209    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f7df\" (UniqueName: \"kubernetes.io/projected/95b9c1f4-445a-48e1-ad63-24fd1e329233-kube-api-access-6f7df\") pod \"payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") " pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.330530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.330500    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:55.607754 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.607721    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"]
Sep 10 21:12:55.609597 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:12:55.609568    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b9c1f4_445a_48e1_ad63_24fd1e329233.slice/crio-a4beafa842c44c16d0af281aba058498e1e74320f1b80b577d1176d97faf2174 WatchSource:0}: Error finding container a4beafa842c44c16d0af281aba058498e1e74320f1b80b577d1176d97faf2174: Status 404 returned error can't find the container with id a4beafa842c44c16d0af281aba058498e1e74320f1b80b577d1176d97faf2174
Sep 10 21:12:55.931360 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:55.931321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:56.266593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:56.266498    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87" event={"ID":"95b9c1f4-445a-48e1-ad63-24fd1e329233","Type":"ContainerStarted","Data":"ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7"}
Sep 10 21:12:56.266593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:56.266574    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:12:56.266789 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:56.266599    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87" event={"ID":"95b9c1f4-445a-48e1-ad63-24fd1e329233","Type":"ContainerStarted","Data":"a4beafa842c44c16d0af281aba058498e1e74320f1b80b577d1176d97faf2174"}
Sep 10 21:12:56.285876 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:56.285809    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87" podStartSLOduration=2.285789318 podStartE2EDuration="2.285789318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:56.285129204 +0000 UTC m=+870.610890126" watchObservedRunningTime="2026-09-10 21:12:56.285789318 +0000 UTC m=+870.611550242"
Sep 10 21:12:56.714382 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:12:56.714342    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:07.274959 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:07.274921    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:13:12.829379 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.829340    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"]
Sep 10 21:13:12.834962 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.834932    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:12.837516 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.837488    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-31e7cb\"/\"kube-root-ca.crt\""
Sep 10 21:13:12.838405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.838379    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-31e7cb\"/\"openshift-service-ca.crt\""
Sep 10 21:13:12.838652 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.838635    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-31e7cb\"/\"test-model-31e7cb-kserve-self-signed-certs\""
Sep 10 21:13:12.838859 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.838840    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-31e7cb\"/\"default-dockercfg-j5chz\""
Sep 10 21:13:12.844249 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.844222    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"]
Sep 10 21:13:12.864066 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.864028    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:12.910378 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.910340    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac9d652-2166-445e-96b7-0ecfc499bbda-tls-certs\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:12.910470 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.910394    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlpcr\" (UniqueName: \"kubernetes.io/projected/bac9d652-2166-445e-96b7-0ecfc499bbda-kube-api-access-xlpcr\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:12.910549 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.910525    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-model-cache\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:12.910609 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.910575    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-dshm\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:12.910669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.910630    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-home\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:12.910723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:12.910700    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-tmp-dir\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.011845 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.011805    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlpcr\" (UniqueName: \"kubernetes.io/projected/bac9d652-2166-445e-96b7-0ecfc499bbda-kube-api-access-xlpcr\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012038 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.011869    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-model-cache\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012038 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.011981    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-dshm\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012160 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.012043    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-home\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012160 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.012079    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-tmp-dir\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012160 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.012136    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac9d652-2166-445e-96b7-0ecfc499bbda-tls-certs\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012313 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.012270    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-model-cache\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012504 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.012482    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-tmp-dir\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.012575 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.012513    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-home\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.014209 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.014189    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-dshm\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.014674 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.014657    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac9d652-2166-445e-96b7-0ecfc499bbda-tls-certs\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.020712 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.020691    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlpcr\" (UniqueName: \"kubernetes.io/projected/bac9d652-2166-445e-96b7-0ecfc499bbda-kube-api-access-xlpcr\") pod \"test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") " pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.164644 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.164607    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:13.439196 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:13.439008    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"]
Sep 10 21:13:14.364150 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:14.364113    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm" event={"ID":"bac9d652-2166-445e-96b7-0ecfc499bbda","Type":"ContainerStarted","Data":"59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad"}
Sep 10 21:13:14.364578 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:14.364164    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm" event={"ID":"bac9d652-2166-445e-96b7-0ecfc499bbda","Type":"ContainerStarted","Data":"66d7f40e9cb0820c277e284b39338142756371dc8ef8fc9d8d01f4e3289900c8"}
Sep 10 21:13:14.386488 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:14.386413    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm" podStartSLOduration=2.386396107 podStartE2EDuration="2.386396107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:14.384401163 +0000 UTC m=+888.710162084" watchObservedRunningTime="2026-09-10 21:13:14.386396107 +0000 UTC m=+888.712157027"
Sep 10 21:13:17.944536 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:17.944494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:19.001354 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:19.001312    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:23.165777 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:23.165737    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:23.166273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:23.165794    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:23.178804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:23.178776    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:23.421927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:23.421839    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:13:24.855537 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:24.855502    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:25.113623 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:25.113532    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:25.226534 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:25.226497    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:25.327500 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:25.327459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:26.161654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:26.161625    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:13:26.162127 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:26.161636    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:13:30.535605 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:30.535568    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:30.642377 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:30.642341    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:33.131379 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:33.131343    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:41.130554 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.130508    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:41.457334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.457285    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:41.498992 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.498960    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"]
Sep 10 21:13:41.499248 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.499219    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87" podUID="95b9c1f4-445a-48e1-ad63-24fd1e329233" containerName="payload-processing" containerID="cri-o://ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7" gracePeriod=30
Sep 10 21:13:41.549674 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.549641    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:41.649449 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.649390    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:41.727753 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.727667    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:41.751946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.751924    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:13:41.790673 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.790641    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c1f4-445a-48e1-ad63-24fd1e329233-plugins-config-volume\") pod \"95b9c1f4-445a-48e1-ad63-24fd1e329233\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") "
Sep 10 21:13:41.790829 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.790774    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f7df\" (UniqueName: \"kubernetes.io/projected/95b9c1f4-445a-48e1-ad63-24fd1e329233-kube-api-access-6f7df\") pod \"95b9c1f4-445a-48e1-ad63-24fd1e329233\" (UID: \"95b9c1f4-445a-48e1-ad63-24fd1e329233\") "
Sep 10 21:13:41.791133 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.791074    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b9c1f4-445a-48e1-ad63-24fd1e329233-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95b9c1f4-445a-48e1-ad63-24fd1e329233" (UID: "95b9c1f4-445a-48e1-ad63-24fd1e329233"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:41.792858 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.792838    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b9c1f4-445a-48e1-ad63-24fd1e329233-kube-api-access-6f7df" (OuterVolumeSpecName: "kube-api-access-6f7df") pod "95b9c1f4-445a-48e1-ad63-24fd1e329233" (UID: "95b9c1f4-445a-48e1-ad63-24fd1e329233"). InnerVolumeSpecName "kube-api-access-6f7df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:41.891931 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.891895    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f7df\" (UniqueName: \"kubernetes.io/projected/95b9c1f4-445a-48e1-ad63-24fd1e329233-kube-api-access-6f7df\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:13:41.891931 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:41.891929    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b9c1f4-445a-48e1-ad63-24fd1e329233-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.059924 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.059839    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:42.515092 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.515060    2584 generic.go:356] "Generic (PLEG): container finished" podID="95b9c1f4-445a-48e1-ad63-24fd1e329233" containerID="ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7" exitCode=0
Sep 10 21:13:42.515510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.515106    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"
Sep 10 21:13:42.515510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.515137    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87" event={"ID":"95b9c1f4-445a-48e1-ad63-24fd1e329233","Type":"ContainerDied","Data":"ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7"}
Sep 10 21:13:42.515510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.515190    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87" event={"ID":"95b9c1f4-445a-48e1-ad63-24fd1e329233","Type":"ContainerDied","Data":"a4beafa842c44c16d0af281aba058498e1e74320f1b80b577d1176d97faf2174"}
Sep 10 21:13:42.515510 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.515217    2584 scope.go:117] "RemoveContainer" containerID="ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7"
Sep 10 21:13:42.523998 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.523981    2584 scope.go:117] "RemoveContainer" containerID="ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7"
Sep 10 21:13:42.524254 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:13:42.524236    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7\": container with ID starting with ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7 not found: ID does not exist" containerID="ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7"
Sep 10 21:13:42.524295 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.524264    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7"} err="failed to get container status \"ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7\": rpc error: code = NotFound desc = could not find container \"ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7\": container with ID starting with ee594936e724b43669ac5bcff9eee8e12323fa0ec4741afab7ca5cbf370181b7 not found: ID does not exist"
Sep 10 21:13:42.533055 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.533029    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"]
Sep 10 21:13:42.536719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:42.536698    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ea71f9-8ffbcf597-2vx87"]
Sep 10 21:13:44.301065 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:44.301032    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b9c1f4-445a-48e1-ad63-24fd1e329233" path="/var/lib/kubelet/pods/95b9c1f4-445a-48e1-ad63-24fd1e329233/volumes"
Sep 10 21:13:47.952731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:47.952694    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:48.058846 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:48.058808    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:48.213100 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:48.213018    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:49.110637 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:49.110599    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:49.211581 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:49.211534    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:49.362800 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:49.362714    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:49.823407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:49.823374    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:50.095778 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:50.095688    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:50.556810 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:50.556771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:51.804232 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:51.804194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:51.903303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:51.903267    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:52.062594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:52.062497    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:57.191750 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:57.191700    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:58.647761 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:58.647721    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:59.405906 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:13:59.405849    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.261767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:01.261731    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.400157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:01.400117    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.590349 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:01.590251    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.687539 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:01.687494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:05.600786 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:05.600750    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:05.678728 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:05.678687    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:05.760859 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:05.760800    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:05.863732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:05.863647    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:06.536887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:06.536838    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:06.610807 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:06.610774    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:06.746598 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:06.746567    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:06.841691 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:06.841596    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:08.234174 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:08.234139    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:14.214388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:14.214348    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:14.315217 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:14.315180    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:14.863661 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:14.863615    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:20.945202 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:20.945160    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:21.042499 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:21.042459    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:21.220847 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:21.220754    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:21.635334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:21.635225    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:22.149567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:22.149530    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:23.820300 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:23.820255    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"]
Sep 10 21:14:24.553278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:24.553241    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:24.663751 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:24.663716    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:24.756504 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:24.756469    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:25.116603 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:25.116560    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"]
Sep 10 21:14:25.138757 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:25.138729    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:25.463767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:25.463735    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:26.298910 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:26.298871    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:26.400198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:26.400167    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:26.477088 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:26.477055    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:26.816763 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:26.816732    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:26.913304 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:26.913261    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:27.008285 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.008250    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"]
Sep 10 21:14:27.008662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.008649    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b9c1f4-445a-48e1-ad63-24fd1e329233" containerName="payload-processing"
Sep 10 21:14:27.008712 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.008665    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b9c1f4-445a-48e1-ad63-24fd1e329233" containerName="payload-processing"
Sep 10 21:14:27.008762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.008753    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="95b9c1f4-445a-48e1-ad63-24fd1e329233" containerName="payload-processing"
Sep 10 21:14:27.013131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.013114    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.016806 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.016785    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c6e2e41d-dockercfg-z5mj7\""
Sep 10 21:14:27.016923 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.016898    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c6e2e41d\""
Sep 10 21:14:27.021367 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.021341    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"]
Sep 10 21:14:27.126895 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.126811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f869d5fa-0e56-4381-a501-18af0804a781-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.127041 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.126909    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hksz8\" (UniqueName: \"kubernetes.io/projected/f869d5fa-0e56-4381-a501-18af0804a781-kube-api-access-hksz8\") pod \"payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.228188 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.228151    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hksz8\" (UniqueName: \"kubernetes.io/projected/f869d5fa-0e56-4381-a501-18af0804a781-kube-api-access-hksz8\") pod \"payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.228369 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.228268    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f869d5fa-0e56-4381-a501-18af0804a781-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.228994 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.228974    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f869d5fa-0e56-4381-a501-18af0804a781-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.238884 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.238858    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hksz8\" (UniqueName: \"kubernetes.io/projected/f869d5fa-0e56-4381-a501-18af0804a781-kube-api-access-hksz8\") pod \"payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.325399 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.325359    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.341288 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.341258    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:27.602390 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.602365    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"]
Sep 10 21:14:27.604767 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:14:27.604725    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf869d5fa_0e56_4381_a501_18af0804a781.slice/crio-119c7f83821cf95d11069a37e4f585f928ef50c3577cf1067af62e7b19f40c82 WatchSource:0}: Error finding container 119c7f83821cf95d11069a37e4f585f928ef50c3577cf1067af62e7b19f40c82: Status 404 returned error can't find the container with id 119c7f83821cf95d11069a37e4f585f928ef50c3577cf1067af62e7b19f40c82
Sep 10 21:14:27.745080 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.745047    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689" event={"ID":"f869d5fa-0e56-4381-a501-18af0804a781","Type":"ContainerStarted","Data":"f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3"}
Sep 10 21:14:27.745246 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.745132    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:27.745246 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.745151    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689" event={"ID":"f869d5fa-0e56-4381-a501-18af0804a781","Type":"ContainerStarted","Data":"119c7f83821cf95d11069a37e4f585f928ef50c3577cf1067af62e7b19f40c82"}
Sep 10 21:14:27.762738 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:27.762682    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689" podStartSLOduration=1.762666119 podStartE2EDuration="1.762666119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:27.761199639 +0000 UTC m=+962.086960549" watchObservedRunningTime="2026-09-10 21:14:27.762666119 +0000 UTC m=+962.088427039"
Sep 10 21:14:28.175647 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:28.175595    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:35.390533 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:35.390493    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:36.004231 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.004192    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"]
Sep 10 21:14:36.004567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.004509    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj" podUID="6f70ac02-5e43-44dc-8b8d-c93f7c801c94" containerName="maas-api" containerID="cri-o://3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c" gracePeriod=30
Sep 10 21:14:36.037847 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.037812    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:36.077078 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.077046    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"]
Sep 10 21:14:36.077319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.077287    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb" podUID="045fa49f-6f8a-444c-9626-aa66ced692a0" containerName="payload-processing" containerID="cri-o://90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b" gracePeriod=30
Sep 10 21:14:36.080436 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.080378    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"]
Sep 10 21:14:36.080634 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.080604    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7" podUID="f6ebc320-d1df-4fbe-8a91-18297bf05234" containerName="payload-pre-processing" containerID="cri-o://d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3" gracePeriod=30
Sep 10 21:14:36.122868 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.122831    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:36.278928 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.278806    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:36.348727 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.348703    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:14:36.388774 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.388698    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:36.399907 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.399883    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:14:36.407776 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.407754    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:14:36.460128 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.460060    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:36.521031 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.520999    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/045fa49f-6f8a-444c-9626-aa66ced692a0-plugins-config-volume\") pod \"045fa49f-6f8a-444c-9626-aa66ced692a0\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") "
Sep 10 21:14:36.521031 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521043    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgvdt\" (UniqueName: \"kubernetes.io/projected/f6ebc320-d1df-4fbe-8a91-18297bf05234-kube-api-access-jgvdt\") pod \"f6ebc320-d1df-4fbe-8a91-18297bf05234\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") "
Sep 10 21:14:36.521290 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521103    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7fbc\" (UniqueName: \"kubernetes.io/projected/045fa49f-6f8a-444c-9626-aa66ced692a0-kube-api-access-m7fbc\") pod \"045fa49f-6f8a-444c-9626-aa66ced692a0\" (UID: \"045fa49f-6f8a-444c-9626-aa66ced692a0\") "
Sep 10 21:14:36.521290 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521134    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-metrics-certs\") pod \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") "
Sep 10 21:14:36.521290 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521177    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmfx4\" (UniqueName: \"kubernetes.io/projected/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-kube-api-access-lmfx4\") pod \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") "
Sep 10 21:14:36.521290 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521286    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-openshift-service-ca\") pod \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") "
Sep 10 21:14:36.521517 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521323    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls\") pod \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\" (UID: \"6f70ac02-5e43-44dc-8b8d-c93f7c801c94\") "
Sep 10 21:14:36.521517 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521350    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ebc320-d1df-4fbe-8a91-18297bf05234-plugins-config-volume\") pod \"f6ebc320-d1df-4fbe-8a91-18297bf05234\" (UID: \"f6ebc320-d1df-4fbe-8a91-18297bf05234\") "
Sep 10 21:14:36.521517 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521467    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/045fa49f-6f8a-444c-9626-aa66ced692a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "045fa49f-6f8a-444c-9626-aa66ced692a0" (UID: "045fa49f-6f8a-444c-9626-aa66ced692a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:36.522002 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521722    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/045fa49f-6f8a-444c-9626-aa66ced692a0-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.522002 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.521773    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f70ac02-5e43-44dc-8b8d-c93f7c801c94" (UID: "6f70ac02-5e43-44dc-8b8d-c93f7c801c94"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:36.522193 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.522058    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6ebc320-d1df-4fbe-8a91-18297bf05234-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6ebc320-d1df-4fbe-8a91-18297bf05234" (UID: "f6ebc320-d1df-4fbe-8a91-18297bf05234"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:36.524081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.524030    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ebc320-d1df-4fbe-8a91-18297bf05234-kube-api-access-jgvdt" (OuterVolumeSpecName: "kube-api-access-jgvdt") pod "f6ebc320-d1df-4fbe-8a91-18297bf05234" (UID: "f6ebc320-d1df-4fbe-8a91-18297bf05234"). InnerVolumeSpecName "kube-api-access-jgvdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:36.524081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.524064    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f70ac02-5e43-44dc-8b8d-c93f7c801c94" (UID: "6f70ac02-5e43-44dc-8b8d-c93f7c801c94"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:36.524245 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.524113    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045fa49f-6f8a-444c-9626-aa66ced692a0-kube-api-access-m7fbc" (OuterVolumeSpecName: "kube-api-access-m7fbc") pod "045fa49f-6f8a-444c-9626-aa66ced692a0" (UID: "045fa49f-6f8a-444c-9626-aa66ced692a0"). InnerVolumeSpecName "kube-api-access-m7fbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:36.524305 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.524288    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f70ac02-5e43-44dc-8b8d-c93f7c801c94" (UID: "6f70ac02-5e43-44dc-8b8d-c93f7c801c94"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:36.524388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.524369    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-kube-api-access-lmfx4" (OuterVolumeSpecName: "kube-api-access-lmfx4") pod "6f70ac02-5e43-44dc-8b8d-c93f7c801c94" (UID: "6f70ac02-5e43-44dc-8b8d-c93f7c801c94"). InnerVolumeSpecName "kube-api-access-lmfx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:36.622654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.622570    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgvdt\" (UniqueName: \"kubernetes.io/projected/f6ebc320-d1df-4fbe-8a91-18297bf05234-kube-api-access-jgvdt\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.622654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.622602    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7fbc\" (UniqueName: \"kubernetes.io/projected/045fa49f-6f8a-444c-9626-aa66ced692a0-kube-api-access-m7fbc\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.622654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.622612    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.622654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.622621    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmfx4\" (UniqueName: \"kubernetes.io/projected/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-kube-api-access-lmfx4\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.622654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.622631    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.622654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.622640    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f70ac02-5e43-44dc-8b8d-c93f7c801c94-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.622654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.622648    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ebc320-d1df-4fbe-8a91-18297bf05234-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:36.751809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.751761    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:36.798574 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.798534    2584 generic.go:356] "Generic (PLEG): container finished" podID="f6ebc320-d1df-4fbe-8a91-18297bf05234" containerID="d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3" exitCode=0
Sep 10 21:14:36.798779 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.798582    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"
Sep 10 21:14:36.798779 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.798627    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7" event={"ID":"f6ebc320-d1df-4fbe-8a91-18297bf05234","Type":"ContainerDied","Data":"d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3"}
Sep 10 21:14:36.798779 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.798677    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7" event={"ID":"f6ebc320-d1df-4fbe-8a91-18297bf05234","Type":"ContainerDied","Data":"c3762c0f14cade4ea77705f2e228665e789385e71c6e8ab7033a5c7965da0b0e"}
Sep 10 21:14:36.798779 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.798699    2584 scope.go:117] "RemoveContainer" containerID="d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3"
Sep 10 21:14:36.800177 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.800149    2584 generic.go:356] "Generic (PLEG): container finished" podID="6f70ac02-5e43-44dc-8b8d-c93f7c801c94" containerID="3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c" exitCode=0
Sep 10 21:14:36.800297 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.800191    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"
Sep 10 21:14:36.800297 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.800246    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj" event={"ID":"6f70ac02-5e43-44dc-8b8d-c93f7c801c94","Type":"ContainerDied","Data":"3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c"}
Sep 10 21:14:36.800297 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.800293    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj" event={"ID":"6f70ac02-5e43-44dc-8b8d-c93f7c801c94","Type":"ContainerDied","Data":"8f11e9a129f89fd7c99fc26fc8c61c984801ec5b8bec11ce52fb2a00478fb693"}
Sep 10 21:14:36.801797 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.801761    2584 generic.go:356] "Generic (PLEG): container finished" podID="045fa49f-6f8a-444c-9626-aa66ced692a0" containerID="90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b" exitCode=0
Sep 10 21:14:36.801913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.801805    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb" event={"ID":"045fa49f-6f8a-444c-9626-aa66ced692a0","Type":"ContainerDied","Data":"90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b"}
Sep 10 21:14:36.801913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.801816    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"
Sep 10 21:14:36.801913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.801840    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb" event={"ID":"045fa49f-6f8a-444c-9626-aa66ced692a0","Type":"ContainerDied","Data":"4f81f35cf1d982da88ad4ad1103c69ee0c3ccb68082ee1c7de93b14b07b0a552"}
Sep 10 21:14:36.809520 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.809498    2584 scope.go:117] "RemoveContainer" containerID="d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3"
Sep 10 21:14:36.809886 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:14:36.809859    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3\": container with ID starting with d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3 not found: ID does not exist" containerID="d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3"
Sep 10 21:14:36.809972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.809897    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3"} err="failed to get container status \"d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3\": rpc error: code = NotFound desc = could not find container \"d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3\": container with ID starting with d6b6dc8588573e1b8458ad4b35b07b5d3ba09e17d47f9d9fc50483cb2ece8df3 not found: ID does not exist"
Sep 10 21:14:36.809972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.809924    2584 scope.go:117] "RemoveContainer" containerID="3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c"
Sep 10 21:14:36.818872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.818853    2584 scope.go:117] "RemoveContainer" containerID="3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c"
Sep 10 21:14:36.819186 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:14:36.819168    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c\": container with ID starting with 3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c not found: ID does not exist" containerID="3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c"
Sep 10 21:14:36.819247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.819194    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c"} err="failed to get container status \"3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c\": rpc error: code = NotFound desc = could not find container \"3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c\": container with ID starting with 3b1fae474fd90dadb212c7771c78ee6037c6e75b9cf2a85a3f6bb9d992722b0c not found: ID does not exist"
Sep 10 21:14:36.819247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.819211    2584 scope.go:117] "RemoveContainer" containerID="90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b"
Sep 10 21:14:36.823992 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.823968    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"]
Sep 10 21:14:36.827236 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.827214    2584 scope.go:117] "RemoveContainer" containerID="90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b"
Sep 10 21:14:36.827518 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:14:36.827495    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b\": container with ID starting with 90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b not found: ID does not exist" containerID="90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b"
Sep 10 21:14:36.827572 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.827526    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b"} err="failed to get container status \"90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b\": rpc error: code = NotFound desc = could not find container \"90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b\": container with ID starting with 90ce8e549f1f449674df4c3a185f2bbdf36ab4e29b9aad4cd8eb9b9cf21c406b not found: ID does not exist"
Sep 10 21:14:36.830437 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.830402    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-76b274-6f96f68b78-jmxs7"]
Sep 10 21:14:36.840599 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.840578    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"]
Sep 10 21:14:36.845319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.845298    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-76b274-5bf58749ff-4pzvj"]
Sep 10 21:14:36.855774 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.855753    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"]
Sep 10 21:14:36.859130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:36.859110    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-76b274-6b8b989b8f-4ssmb"]
Sep 10 21:14:38.301136 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:38.301087    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="045fa49f-6f8a-444c-9626-aa66ced692a0" path="/var/lib/kubelet/pods/045fa49f-6f8a-444c-9626-aa66ced692a0/volumes"
Sep 10 21:14:38.301616 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:38.301547    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f70ac02-5e43-44dc-8b8d-c93f7c801c94" path="/var/lib/kubelet/pods/6f70ac02-5e43-44dc-8b8d-c93f7c801c94/volumes"
Sep 10 21:14:38.302065 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:38.302046    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6ebc320-d1df-4fbe-8a91-18297bf05234" path="/var/lib/kubelet/pods/f6ebc320-d1df-4fbe-8a91-18297bf05234/volumes"
Sep 10 21:14:38.753072 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:38.753043    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:14:38.840053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:38.840008    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf" podUID="e5a6b291-05eb-4ab0-ae13-57206bb057f5" containerName="main" containerID="cri-o://0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078" gracePeriod=45
Sep 10 21:14:39.104751 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.104720    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:14:39.246580 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.246542    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5slw\" (UniqueName: \"kubernetes.io/projected/e5a6b291-05eb-4ab0-ae13-57206bb057f5-kube-api-access-r5slw\") pod \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") "
Sep 10 21:14:39.246752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.246604    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-model-cache\") pod \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") "
Sep 10 21:14:39.246752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.246646    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tls-certs\") pod \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") "
Sep 10 21:14:39.246752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.246707    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-home\") pod \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") "
Sep 10 21:14:39.246752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.246726    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-dshm\") pod \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") "
Sep 10 21:14:39.246752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.246746    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tmp-dir\") pod \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\" (UID: \"e5a6b291-05eb-4ab0-ae13-57206bb057f5\") "
Sep 10 21:14:39.247024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.246882    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "e5a6b291-05eb-4ab0-ae13-57206bb057f5" (UID: "e5a6b291-05eb-4ab0-ae13-57206bb057f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:39.247096 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.247020    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-home" (OuterVolumeSpecName: "home") pod "e5a6b291-05eb-4ab0-ae13-57206bb057f5" (UID: "e5a6b291-05eb-4ab0-ae13-57206bb057f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:39.247096 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.247077    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e5a6b291-05eb-4ab0-ae13-57206bb057f5" (UID: "e5a6b291-05eb-4ab0-ae13-57206bb057f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:39.247096 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.247089    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-model-cache\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:39.248922 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.248896    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-dshm" (OuterVolumeSpecName: "dshm") pod "e5a6b291-05eb-4ab0-ae13-57206bb057f5" (UID: "e5a6b291-05eb-4ab0-ae13-57206bb057f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:39.249029 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.248984    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e5a6b291-05eb-4ab0-ae13-57206bb057f5" (UID: "e5a6b291-05eb-4ab0-ae13-57206bb057f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:39.249029 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.248988    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5a6b291-05eb-4ab0-ae13-57206bb057f5-kube-api-access-r5slw" (OuterVolumeSpecName: "kube-api-access-r5slw") pod "e5a6b291-05eb-4ab0-ae13-57206bb057f5" (UID: "e5a6b291-05eb-4ab0-ae13-57206bb057f5"). InnerVolumeSpecName "kube-api-access-r5slw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:39.347896 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.347805    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-home\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:39.347896 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.347836    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-dshm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:39.347896 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.347847    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tmp-dir\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:39.347896 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.347858    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5slw\" (UniqueName: \"kubernetes.io/projected/e5a6b291-05eb-4ab0-ae13-57206bb057f5-kube-api-access-r5slw\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:39.347896 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.347866    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5a6b291-05eb-4ab0-ae13-57206bb057f5-tls-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:39.822125 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.822083    2584 generic.go:356] "Generic (PLEG): container finished" podID="e5a6b291-05eb-4ab0-ae13-57206bb057f5" containerID="0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078" exitCode=0
Sep 10 21:14:39.822125 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.822124    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"
Sep 10 21:14:39.822343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.822170    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf" event={"ID":"e5a6b291-05eb-4ab0-ae13-57206bb057f5","Type":"ContainerDied","Data":"0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078"}
Sep 10 21:14:39.822343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.822214    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf" event={"ID":"e5a6b291-05eb-4ab0-ae13-57206bb057f5","Type":"ContainerDied","Data":"69233ed8c57b6a27ac4d3bbef17ca075e4f029c7f4b8d7b6d1ac40b6cbffb0d9"}
Sep 10 21:14:39.822343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.822231    2584 scope.go:117] "RemoveContainer" containerID="0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078"
Sep 10 21:14:39.831251 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.831234    2584 scope.go:117] "RemoveContainer" containerID="0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078"
Sep 10 21:14:39.831565 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:14:39.831542    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078\": container with ID starting with 0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078 not found: ID does not exist" containerID="0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078"
Sep 10 21:14:39.831664 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.831570    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078"} err="failed to get container status \"0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078\": rpc error: code = NotFound desc = could not find container \"0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078\": container with ID starting with 0da0e13d4ba1fe3bfcce69a309bcdd75864392285fc4fc6463947203ec1ae078 not found: ID does not exist"
Sep 10 21:14:39.846129 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.846097    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"]
Sep 10 21:14:39.851845 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:39.851819    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-76b274/test-model-76b274-kserve-69d45fcb89-l7lwf"]
Sep 10 21:14:40.135665 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.135565    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm" podUID="bac9d652-2166-445e-96b7-0ecfc499bbda" containerName="main" containerID="cri-o://59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad" gracePeriod=45
Sep 10 21:14:40.302361 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.302319    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5a6b291-05eb-4ab0-ae13-57206bb057f5" path="/var/lib/kubelet/pods/e5a6b291-05eb-4ab0-ae13-57206bb057f5/volumes"
Sep 10 21:14:40.393521 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.393453    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:14:40.558433 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558383    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac9d652-2166-445e-96b7-0ecfc499bbda-tls-certs\") pod \"bac9d652-2166-445e-96b7-0ecfc499bbda\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") "
Sep 10 21:14:40.558641 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558505    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlpcr\" (UniqueName: \"kubernetes.io/projected/bac9d652-2166-445e-96b7-0ecfc499bbda-kube-api-access-xlpcr\") pod \"bac9d652-2166-445e-96b7-0ecfc499bbda\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") "
Sep 10 21:14:40.558641 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558564    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-dshm\") pod \"bac9d652-2166-445e-96b7-0ecfc499bbda\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") "
Sep 10 21:14:40.558641 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558615    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-model-cache\") pod \"bac9d652-2166-445e-96b7-0ecfc499bbda\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") "
Sep 10 21:14:40.558641 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558638    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-tmp-dir\") pod \"bac9d652-2166-445e-96b7-0ecfc499bbda\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") "
Sep 10 21:14:40.558841 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558661    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-home\") pod \"bac9d652-2166-445e-96b7-0ecfc499bbda\" (UID: \"bac9d652-2166-445e-96b7-0ecfc499bbda\") "
Sep 10 21:14:40.558919 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558888    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bac9d652-2166-445e-96b7-0ecfc499bbda" (UID: "bac9d652-2166-445e-96b7-0ecfc499bbda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:40.558971 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558945    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-home" (OuterVolumeSpecName: "home") pod "bac9d652-2166-445e-96b7-0ecfc499bbda" (UID: "bac9d652-2166-445e-96b7-0ecfc499bbda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:40.559010 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.558903    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-model-cache" (OuterVolumeSpecName: "model-cache") pod "bac9d652-2166-445e-96b7-0ecfc499bbda" (UID: "bac9d652-2166-445e-96b7-0ecfc499bbda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:40.560756 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.560729    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-dshm" (OuterVolumeSpecName: "dshm") pod "bac9d652-2166-445e-96b7-0ecfc499bbda" (UID: "bac9d652-2166-445e-96b7-0ecfc499bbda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:40.560895 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.560829    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac9d652-2166-445e-96b7-0ecfc499bbda-kube-api-access-xlpcr" (OuterVolumeSpecName: "kube-api-access-xlpcr") pod "bac9d652-2166-445e-96b7-0ecfc499bbda" (UID: "bac9d652-2166-445e-96b7-0ecfc499bbda"). InnerVolumeSpecName "kube-api-access-xlpcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:40.560895 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.560844    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac9d652-2166-445e-96b7-0ecfc499bbda-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bac9d652-2166-445e-96b7-0ecfc499bbda" (UID: "bac9d652-2166-445e-96b7-0ecfc499bbda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:40.659880 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.659836    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlpcr\" (UniqueName: \"kubernetes.io/projected/bac9d652-2166-445e-96b7-0ecfc499bbda-kube-api-access-xlpcr\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:40.659880 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.659874    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-dshm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:40.659880 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.659885    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-model-cache\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:40.660129 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.659895    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-tmp-dir\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:40.660129 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.659902    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac9d652-2166-445e-96b7-0ecfc499bbda-home\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:40.660129 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.659910    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac9d652-2166-445e-96b7-0ecfc499bbda-tls-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:40.829781 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.829745    2584 generic.go:356] "Generic (PLEG): container finished" podID="bac9d652-2166-445e-96b7-0ecfc499bbda" containerID="59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad" exitCode=0
Sep 10 21:14:40.829965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.829793    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"
Sep 10 21:14:40.829965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.829801    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm" event={"ID":"bac9d652-2166-445e-96b7-0ecfc499bbda","Type":"ContainerDied","Data":"59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad"}
Sep 10 21:14:40.829965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.829839    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm" event={"ID":"bac9d652-2166-445e-96b7-0ecfc499bbda","Type":"ContainerDied","Data":"66d7f40e9cb0820c277e284b39338142756371dc8ef8fc9d8d01f4e3289900c8"}
Sep 10 21:14:40.829965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.829862    2584 scope.go:117] "RemoveContainer" containerID="59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad"
Sep 10 21:14:40.838943 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.838921    2584 scope.go:117] "RemoveContainer" containerID="59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad"
Sep 10 21:14:40.839266 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:14:40.839241    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad\": container with ID starting with 59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad not found: ID does not exist" containerID="59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad"
Sep 10 21:14:40.839345 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.839278    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad"} err="failed to get container status \"59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad\": rpc error: code = NotFound desc = could not find container \"59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad\": container with ID starting with 59e6d3a178ece0f4dc4e939068cb76e01030081673d7d44ec778ec58a891ccad not found: ID does not exist"
Sep 10 21:14:40.859348 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.859314    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"]
Sep 10 21:14:40.861081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:40.861051    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-31e7cb/test-model-31e7cb-kserve-6b4d8dd6c5-vmnrm"]
Sep 10 21:14:42.301454 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:42.301400    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac9d652-2166-445e-96b7-0ecfc499bbda" path="/var/lib/kubelet/pods/bac9d652-2166-445e-96b7-0ecfc499bbda/volumes"
Sep 10 21:14:43.694267 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:43.694231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:43.795394 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:43.795358    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:43.942993 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:43.942953    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:44.659849 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:44.659799    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:46.137452 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:46.137395    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:47.364301 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:47.364262    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:49.828627 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:49.828539    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:50.140755 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:50.140643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:50.240910 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:50.240862    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:50.577870 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:50.577833    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:54.443357 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:54.443313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.138864 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.138832    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"]
Sep 10 21:14:55.139099 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.139077    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k" podUID="9c31fd1f-8234-427a-a1b0-8c864cf1497a" containerName="maas-api" containerID="cri-o://b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155" gracePeriod=30
Sep 10 21:14:55.176483 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.176447    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.221044 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.221012    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"]
Sep 10 21:14:55.221280 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.221251    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt" podUID="bd17a102-039d-41c9-93b6-6ff691112ac4" containerName="payload-pre-processing" containerID="cri-o://8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3" gracePeriod=30
Sep 10 21:14:55.276052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.275980    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.377094 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.377023    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.463601 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.463561    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.480104 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.480077    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:14:55.486950 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.486917    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls\") pod \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") "
Sep 10 21:14:55.486950 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.486951    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch7pr\" (UniqueName: \"kubernetes.io/projected/9c31fd1f-8234-427a-a1b0-8c864cf1497a-kube-api-access-ch7pr\") pod \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") "
Sep 10 21:14:55.487177 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.486982    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-metrics-certs\") pod \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") "
Sep 10 21:14:55.487177 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.487012    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c31fd1f-8234-427a-a1b0-8c864cf1497a-openshift-service-ca\") pod \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\" (UID: \"9c31fd1f-8234-427a-a1b0-8c864cf1497a\") "
Sep 10 21:14:55.487482 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.487442    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c31fd1f-8234-427a-a1b0-8c864cf1497a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c31fd1f-8234-427a-a1b0-8c864cf1497a" (UID: "9c31fd1f-8234-427a-a1b0-8c864cf1497a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:55.489712 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.489685    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c31fd1f-8234-427a-a1b0-8c864cf1497a" (UID: "9c31fd1f-8234-427a-a1b0-8c864cf1497a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:55.489804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.489735    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9c31fd1f-8234-427a-a1b0-8c864cf1497a" (UID: "9c31fd1f-8234-427a-a1b0-8c864cf1497a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:55.489804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.489756    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c31fd1f-8234-427a-a1b0-8c864cf1497a-kube-api-access-ch7pr" (OuterVolumeSpecName: "kube-api-access-ch7pr") pod "9c31fd1f-8234-427a-a1b0-8c864cf1497a" (UID: "9c31fd1f-8234-427a-a1b0-8c864cf1497a"). InnerVolumeSpecName "kube-api-access-ch7pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:55.506175 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.506141    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:14:55.588222 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588190    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd17a102-039d-41c9-93b6-6ff691112ac4-plugins-config-volume\") pod \"bd17a102-039d-41c9-93b6-6ff691112ac4\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") "
Sep 10 21:14:55.588411 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588233    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbl58\" (UniqueName: \"kubernetes.io/projected/bd17a102-039d-41c9-93b6-6ff691112ac4-kube-api-access-lbl58\") pod \"bd17a102-039d-41c9-93b6-6ff691112ac4\" (UID: \"bd17a102-039d-41c9-93b6-6ff691112ac4\") "
Sep 10 21:14:55.588636 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588606    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd17a102-039d-41c9-93b6-6ff691112ac4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd17a102-039d-41c9-93b6-6ff691112ac4" (UID: "bd17a102-039d-41c9-93b6-6ff691112ac4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:55.588724 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588709    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd17a102-039d-41c9-93b6-6ff691112ac4-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.588762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588727    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.588762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588739    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch7pr\" (UniqueName: \"kubernetes.io/projected/9c31fd1f-8234-427a-a1b0-8c864cf1497a-kube-api-access-ch7pr\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.588762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588747    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c31fd1f-8234-427a-a1b0-8c864cf1497a-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.588762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.588755    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c31fd1f-8234-427a-a1b0-8c864cf1497a-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.590314 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.590285    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd17a102-039d-41c9-93b6-6ff691112ac4-kube-api-access-lbl58" (OuterVolumeSpecName: "kube-api-access-lbl58") pod "bd17a102-039d-41c9-93b6-6ff691112ac4" (UID: "bd17a102-039d-41c9-93b6-6ff691112ac4"). InnerVolumeSpecName "kube-api-access-lbl58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:55.689257 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.689170    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbl58\" (UniqueName: \"kubernetes.io/projected/bd17a102-039d-41c9-93b6-6ff691112ac4-kube-api-access-lbl58\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.807160 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.807116    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.915198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.915166    2584 generic.go:356] "Generic (PLEG): container finished" podID="9c31fd1f-8234-427a-a1b0-8c864cf1497a" containerID="b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155" exitCode=0
Sep 10 21:14:55.915350 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.915210    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"
Sep 10 21:14:55.915350 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.915241    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k" event={"ID":"9c31fd1f-8234-427a-a1b0-8c864cf1497a","Type":"ContainerDied","Data":"b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155"}
Sep 10 21:14:55.915350 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.915289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k" event={"ID":"9c31fd1f-8234-427a-a1b0-8c864cf1497a","Type":"ContainerDied","Data":"8e105d6dc5a65d8aa0467b2eb2cb983aa672fd14209d55acabac4c4d4d5510c5"}
Sep 10 21:14:55.915350 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.915317    2584 scope.go:117] "RemoveContainer" containerID="b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155"
Sep 10 21:14:55.916693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.916671    2584 generic.go:356] "Generic (PLEG): container finished" podID="bd17a102-039d-41c9-93b6-6ff691112ac4" containerID="8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3" exitCode=0
Sep 10 21:14:55.916778 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.916698    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt" event={"ID":"bd17a102-039d-41c9-93b6-6ff691112ac4","Type":"ContainerDied","Data":"8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3"}
Sep 10 21:14:55.916778 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.916716    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"
Sep 10 21:14:55.916778 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.916739    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt" event={"ID":"bd17a102-039d-41c9-93b6-6ff691112ac4","Type":"ContainerDied","Data":"d5c1c9e15512ab01e9b02aecf60d79dd6d9b377d68c3481db4ce3e8c3d704f89"}
Sep 10 21:14:55.927256 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.927220    2584 scope.go:117] "RemoveContainer" containerID="b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155"
Sep 10 21:14:55.928160 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:14:55.928132    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155\": container with ID starting with b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155 not found: ID does not exist" containerID="b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155"
Sep 10 21:14:55.928282 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.928170    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155"} err="failed to get container status \"b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155\": rpc error: code = NotFound desc = could not find container \"b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155\": container with ID starting with b96ef6c67e8954794432571a3df83afd6f80c9524b4e9a3c4992416de3df9155 not found: ID does not exist"
Sep 10 21:14:55.928282 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.928195    2584 scope.go:117] "RemoveContainer" containerID="8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3"
Sep 10 21:14:55.936713 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.936697    2584 scope.go:117] "RemoveContainer" containerID="8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3"
Sep 10 21:14:55.936962 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:14:55.936944    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3\": container with ID starting with 8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3 not found: ID does not exist" containerID="8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3"
Sep 10 21:14:55.937008 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.936970    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3"} err="failed to get container status \"8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3\": rpc error: code = NotFound desc = could not find container \"8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3\": container with ID starting with 8ec7b6700d097465be32d1fde110176a802f39a4185fbaa515be2b064a8541d3 not found: ID does not exist"
Sep 10 21:14:55.943599 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.943534    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"]
Sep 10 21:14:55.948616 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.948589    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-31e7cb-75d5d47d8c-b947k"]
Sep 10 21:14:55.963071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.963041    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"]
Sep 10 21:14:55.973172 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:55.973142    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-31e7cb-595db48894-lvfvt"]
Sep 10 21:14:56.307288 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:56.307205    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c31fd1f-8234-427a-a1b0-8c864cf1497a" path="/var/lib/kubelet/pods/9c31fd1f-8234-427a-a1b0-8c864cf1497a/volumes"
Sep 10 21:14:56.307600 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:56.307585    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd17a102-039d-41c9-93b6-6ff691112ac4" path="/var/lib/kubelet/pods/bd17a102-039d-41c9-93b6-6ff691112ac4/volumes"
Sep 10 21:14:57.112084 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:57.112034    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:57.275702 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:57.275662    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:57.690211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:14:57.690171    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:00.143163 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143130    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"]
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143502    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5a6b291-05eb-4ab0-ae13-57206bb057f5" containerName="main"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143515    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5a6b291-05eb-4ab0-ae13-57206bb057f5" containerName="main"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143525    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f70ac02-5e43-44dc-8b8d-c93f7c801c94" containerName="maas-api"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143531    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f70ac02-5e43-44dc-8b8d-c93f7c801c94" containerName="maas-api"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143540    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c31fd1f-8234-427a-a1b0-8c864cf1497a" containerName="maas-api"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143546    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c31fd1f-8234-427a-a1b0-8c864cf1497a" containerName="maas-api"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143561    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac9d652-2166-445e-96b7-0ecfc499bbda" containerName="main"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143566    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac9d652-2166-445e-96b7-0ecfc499bbda" containerName="main"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143574    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd17a102-039d-41c9-93b6-6ff691112ac4" containerName="payload-pre-processing"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143580    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd17a102-039d-41c9-93b6-6ff691112ac4" containerName="payload-pre-processing"
Sep 10 21:15:00.143585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143589    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045fa49f-6f8a-444c-9626-aa66ced692a0" containerName="payload-processing"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143595    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="045fa49f-6f8a-444c-9626-aa66ced692a0" containerName="payload-processing"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143601    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6ebc320-d1df-4fbe-8a91-18297bf05234" containerName="payload-pre-processing"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143606    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6ebc320-d1df-4fbe-8a91-18297bf05234" containerName="payload-pre-processing"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143672    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="045fa49f-6f8a-444c-9626-aa66ced692a0" containerName="payload-processing"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143679    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6ebc320-d1df-4fbe-8a91-18297bf05234" containerName="payload-pre-processing"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143685    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c31fd1f-8234-427a-a1b0-8c864cf1497a" containerName="maas-api"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143690    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac9d652-2166-445e-96b7-0ecfc499bbda" containerName="main"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143695    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f70ac02-5e43-44dc-8b8d-c93f7c801c94" containerName="maas-api"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143702    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5a6b291-05eb-4ab0-ae13-57206bb057f5" containerName="main"
Sep 10 21:15:00.143980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.143707    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd17a102-039d-41c9-93b6-6ff691112ac4" containerName="payload-pre-processing"
Sep 10 21:15:00.148072 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.148049    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"
Sep 10 21:15:00.150762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.150737    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8q9zm\""
Sep 10 21:15:00.155146 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.155107    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"]
Sep 10 21:15:00.161844 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.161823    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"
Sep 10 21:15:00.447658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.447632    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"]
Sep 10 21:15:00.450291 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:15:00.450264    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51211e8_2db4_41eb_bc10_b882e1de5bc7.slice/crio-22e3450f1170a3c05d28e74c1656a1c129c0ba6a6d4ed28e6313f4ee724da524 WatchSource:0}: Error finding container 22e3450f1170a3c05d28e74c1656a1c129c0ba6a6d4ed28e6313f4ee724da524: Status 404 returned error can't find the container with id 22e3450f1170a3c05d28e74c1656a1c129c0ba6a6d4ed28e6313f4ee724da524
Sep 10 21:15:00.943547 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:00.943512    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j" event={"ID":"e51211e8-2db4-41eb-bc10-b882e1de5bc7","Type":"ContainerStarted","Data":"22e3450f1170a3c05d28e74c1656a1c129c0ba6a6d4ed28e6313f4ee724da524"}
Sep 10 21:15:02.418617 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:02.418570    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:02.656475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:02.656438    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:02.753597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:02.753549    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:02.957083 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:02.956989    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j" event={"ID":"e51211e8-2db4-41eb-bc10-b882e1de5bc7","Type":"ContainerStarted","Data":"9a5a03d7c05f1dfbdec4468fb8700157ca534d9ec600d71fb57d68762fd044cf"}
Sep 10 21:15:02.976998 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:02.976942    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j" podStartSLOduration=0.962162175 podStartE2EDuration="2.976925874s" totalImagesPullingTime="2.014763699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:02.974604452 +0000 UTC m=+997.300365375" watchObservedRunningTime="2026-09-10 21:15:02.976925874 +0000 UTC m=+997.302686794"
Sep 10 21:15:03.211609 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.211522    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:03.478326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.478252    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"]
Sep 10 21:15:03.483147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.483124    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.485849 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.485826    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:15:03.485960 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.485827    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-14acb0-openshift-default-dockercfg-vwbtg\""
Sep 10 21:15:03.495132 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.495099    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"]
Sep 10 21:15:03.509433 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.509393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:03.560842 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.560801    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdqnl\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-kube-api-access-gdqnl\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561006 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.560849    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-data\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561006 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.560906    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istiod-ca-cert\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561006 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.560932    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-envoy\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.561032    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-credential-socket\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.561081    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-certs\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.561130    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-token\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561270 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.561161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-socket\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.561270 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.561179    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-podinfo\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.610496 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.610455    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:03.662300 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662263    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-socket\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662300 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662299    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-podinfo\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662323    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdqnl\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-kube-api-access-gdqnl\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662344    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-data\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662403    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istiod-ca-cert\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662473    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-envoy\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662537    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-credential-socket\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662592    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-certs\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662919 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662651    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-token\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.662919 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662747    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-socket\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.663023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662912    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-data\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.663023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.662954    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-credential-socket\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.663121 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.663049    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-certs\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.663167 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.663140    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istiod-ca-cert\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.664776 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.664753    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-envoy\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.664982 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.664967    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-podinfo\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.672069 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.672045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-token\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.672280 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.672258    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdqnl\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-kube-api-access-gdqnl\") pod \"e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") " pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.834497 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.834378    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:03.964679 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.964639    2584 generic.go:356] "Generic (PLEG): container finished" podID="e51211e8-2db4-41eb-bc10-b882e1de5bc7" containerID="9a5a03d7c05f1dfbdec4468fb8700157ca534d9ec600d71fb57d68762fd044cf" exitCode=0
Sep 10 21:15:03.964879 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:03.964723    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j" event={"ID":"e51211e8-2db4-41eb-bc10-b882e1de5bc7","Type":"ContainerDied","Data":"9a5a03d7c05f1dfbdec4468fb8700157ca534d9ec600d71fb57d68762fd044cf"}
Sep 10 21:15:04.119535 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:04.119502    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"]
Sep 10 21:15:04.121578 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:15:04.121546    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fdb282_b518_41ab_b67a_75bf7c41ec27.slice/crio-e72c489d05bf9846090bb83ac30c27d4e0f93183aaf08b24a596c2e12e97fcd7 WatchSource:0}: Error finding container e72c489d05bf9846090bb83ac30c27d4e0f93183aaf08b24a596c2e12e97fcd7: Status 404 returned error can't find the container with id e72c489d05bf9846090bb83ac30c27d4e0f93183aaf08b24a596c2e12e97fcd7
Sep 10 21:15:04.971447 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:04.971387    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" event={"ID":"e7fdb282-b518-41ab-b67a-75bf7c41ec27","Type":"ContainerStarted","Data":"e72c489d05bf9846090bb83ac30c27d4e0f93183aaf08b24a596c2e12e97fcd7"}
Sep 10 21:15:05.110370 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:05.110343    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"
Sep 10 21:15:05.980575 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:05.980544    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"
Sep 10 21:15:05.981046 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:05.980547    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j" event={"ID":"e51211e8-2db4-41eb-bc10-b882e1de5bc7","Type":"ContainerDied","Data":"22e3450f1170a3c05d28e74c1656a1c129c0ba6a6d4ed28e6313f4ee724da524"}
Sep 10 21:15:05.981046 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:05.980680    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22e3450f1170a3c05d28e74c1656a1c129c0ba6a6d4ed28e6313f4ee724da524"
Sep 10 21:15:07.427112 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:07.427076    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:15:07.427373 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:07.427146    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:15:07.427373 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:07.427176    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:15:07.994457 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:07.994404    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" event={"ID":"e7fdb282-b518-41ab-b67a-75bf7c41ec27","Type":"ContainerStarted","Data":"a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0"}
Sep 10 21:15:08.017197 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.017142    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podStartSLOduration=1.714169729 podStartE2EDuration="5.01712477s" totalImagesPullingTime="3.302955041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:08.01517563 +0000 UTC m=+1002.340936552" watchObservedRunningTime="2026-09-10 21:15:08.01712477 +0000 UTC m=+1002.342885691"
Sep 10 21:15:08.156447 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.156377    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"]
Sep 10 21:15:08.156936 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.156900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:08.158716 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.158691    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c6e2e41d-29817915-bnp7j"]
Sep 10 21:15:08.197157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.197119    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"]
Sep 10 21:15:08.197367 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.197336    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689" podUID="f869d5fa-0e56-4381-a501-18af0804a781" containerName="payload-pre-processing" containerID="cri-o://f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3" gracePeriod=30
Sep 10 21:15:08.243437 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.243379    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:08.301616 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.301520    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e51211e8-2db4-41eb-bc10-b882e1de5bc7" path="/var/lib/kubelet/pods/e51211e8-2db4-41eb-bc10-b882e1de5bc7/volumes"
Sep 10 21:15:08.344698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.344656    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:08.415441 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.415397    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:08.457014 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.456989    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:15:08.509968 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.509935    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f869d5fa-0e56-4381-a501-18af0804a781-plugins-config-volume\") pod \"f869d5fa-0e56-4381-a501-18af0804a781\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") "
Sep 10 21:15:08.510137 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.509973    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hksz8\" (UniqueName: \"kubernetes.io/projected/f869d5fa-0e56-4381-a501-18af0804a781-kube-api-access-hksz8\") pod \"f869d5fa-0e56-4381-a501-18af0804a781\" (UID: \"f869d5fa-0e56-4381-a501-18af0804a781\") "
Sep 10 21:15:08.510303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.510281    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f869d5fa-0e56-4381-a501-18af0804a781-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f869d5fa-0e56-4381-a501-18af0804a781" (UID: "f869d5fa-0e56-4381-a501-18af0804a781"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:08.512081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.512062    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f869d5fa-0e56-4381-a501-18af0804a781-kube-api-access-hksz8" (OuterVolumeSpecName: "kube-api-access-hksz8") pod "f869d5fa-0e56-4381-a501-18af0804a781" (UID: "f869d5fa-0e56-4381-a501-18af0804a781"). InnerVolumeSpecName "kube-api-access-hksz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:15:08.610887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.610804    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hksz8\" (UniqueName: \"kubernetes.io/projected/f869d5fa-0e56-4381-a501-18af0804a781-kube-api-access-hksz8\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.610887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.610831    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f869d5fa-0e56-4381-a501-18af0804a781-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.734104 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.734066    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:08.834913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.834873    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:08.840103 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:08.840072    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:09.001157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.001121    2584 generic.go:356] "Generic (PLEG): container finished" podID="f869d5fa-0e56-4381-a501-18af0804a781" containerID="f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3" exitCode=0
Sep 10 21:15:09.001157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.001161    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"
Sep 10 21:15:09.001473 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.001217    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689" event={"ID":"f869d5fa-0e56-4381-a501-18af0804a781","Type":"ContainerDied","Data":"f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3"}
Sep 10 21:15:09.001473 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.001270    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689" event={"ID":"f869d5fa-0e56-4381-a501-18af0804a781","Type":"ContainerDied","Data":"119c7f83821cf95d11069a37e4f585f928ef50c3577cf1067af62e7b19f40c82"}
Sep 10 21:15:09.001473 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.001298    2584 scope.go:117] "RemoveContainer" containerID="f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3"
Sep 10 21:15:09.001633 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.001609    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:09.002700 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.002681    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:09.010493 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.010468    2584 scope.go:117] "RemoveContainer" containerID="f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3"
Sep 10 21:15:09.010822 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:15:09.010803    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3\": container with ID starting with f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3 not found: ID does not exist" containerID="f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3"
Sep 10 21:15:09.010890 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.010831    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3"} err="failed to get container status \"f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3\": rpc error: code = NotFound desc = could not find container \"f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3\": container with ID starting with f749ee765b9a88df2fe20129179d55dd3c4b7b9488d392be0c586fbc9a51bbd3 not found: ID does not exist"
Sep 10 21:15:09.047299 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.047266    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"]
Sep 10 21:15:09.052103 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:09.052056    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-c6e2e41d-69bbf46cfd-cf689"]
Sep 10 21:15:10.219575 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:10.219535    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:10.301847 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:10.301812    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f869d5fa-0e56-4381-a501-18af0804a781" path="/var/lib/kubelet/pods/f869d5fa-0e56-4381-a501-18af0804a781/volumes"
Sep 10 21:15:10.315593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:10.315562    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:11.353360 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.353261    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"]
Sep 10 21:15:11.354642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.354482    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e51211e8-2db4-41eb-bc10-b882e1de5bc7" containerName="cleanup"
Sep 10 21:15:11.354642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.354506    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51211e8-2db4-41eb-bc10-b882e1de5bc7" containerName="cleanup"
Sep 10 21:15:11.354642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.354529    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f869d5fa-0e56-4381-a501-18af0804a781" containerName="payload-pre-processing"
Sep 10 21:15:11.354642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.354538    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f869d5fa-0e56-4381-a501-18af0804a781" containerName="payload-pre-processing"
Sep 10 21:15:11.354642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.354639    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f869d5fa-0e56-4381-a501-18af0804a781" containerName="payload-pre-processing"
Sep 10 21:15:11.354979 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.354653    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e51211e8-2db4-41eb-bc10-b882e1de5bc7" containerName="cleanup"
Sep 10 21:15:11.359356 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.359327    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.361946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.361919    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-14acb0-serving-cert\""
Sep 10 21:15:11.369840 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.369806    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"]
Sep 10 21:15:11.395857 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.395818    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:11.436363 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.436328    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb022a71-1c63-4622-af8a-2bd0a42eddec-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.436556 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.436372    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.436556 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.436515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wglcr\" (UniqueName: \"kubernetes.io/projected/fb022a71-1c63-4622-af8a-2bd0a42eddec-kube-api-access-wglcr\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.436556 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.436551    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.496702 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.496598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:11.538174 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.538138    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wglcr\" (UniqueName: \"kubernetes.io/projected/fb022a71-1c63-4622-af8a-2bd0a42eddec-kube-api-access-wglcr\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.538339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.538200    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.538339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.538259    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb022a71-1c63-4622-af8a-2bd0a42eddec-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.538339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.538290    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.539031 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.539006    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb022a71-1c63-4622-af8a-2bd0a42eddec-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.540704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.540681    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.540859 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.540836    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.547160 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.547132    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wglcr\" (UniqueName: \"kubernetes.io/projected/fb022a71-1c63-4622-af8a-2bd0a42eddec-kube-api-access-wglcr\") pod \"maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.595119 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.595080    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:11.672412 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.672371    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:11.950003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.949979    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"]
Sep 10 21:15:11.951987 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:15:11.951936    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb022a71_1c63_4622_af8a_2bd0a42eddec.slice/crio-194845290f9caa8af7510ed46dc627c79047daea95451b05f9f7ee959bf269f5 WatchSource:0}: Error finding container 194845290f9caa8af7510ed46dc627c79047daea95451b05f9f7ee959bf269f5: Status 404 returned error can't find the container with id 194845290f9caa8af7510ed46dc627c79047daea95451b05f9f7ee959bf269f5
Sep 10 21:15:11.953646 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:11.953626    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:15:12.010367 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.010334    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:12.018657 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.018629    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt" event={"ID":"fb022a71-1c63-4622-af8a-2bd0a42eddec","Type":"ContainerStarted","Data":"194845290f9caa8af7510ed46dc627c79047daea95451b05f9f7ee959bf269f5"}
Sep 10 21:15:12.109745 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.109707    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:12.191031 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.190993    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"]
Sep 10 21:15:12.196328 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.196302    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.198655 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.198631    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-14acb0\""
Sep 10 21:15:12.198767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.198661    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-14acb0-dockercfg-cqmp9\""
Sep 10 21:15:12.206044 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.206011    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"]
Sep 10 21:15:12.246428 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.246371    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.246640 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.246453    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st6cp\" (UniqueName: \"kubernetes.io/projected/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-kube-api-access-st6cp\") pod \"payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.347405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.347372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.347694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.347445    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st6cp\" (UniqueName: \"kubernetes.io/projected/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-kube-api-access-st6cp\") pod \"payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.348311 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.348286    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.359669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.359641    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st6cp\" (UniqueName: \"kubernetes.io/projected/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-kube-api-access-st6cp\") pod \"payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.508720 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.508645    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:12.785059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:12.784994    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"]
Sep 10 21:15:12.787779 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:15:12.787745    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b1c2bb_15b5_4933_8dd4_4a7406ace11e.slice/crio-da6d778427ab8cdf20f0a516292328dcb1cc3ba3790ad467c49a6faa548aae0e WatchSource:0}: Error finding container da6d778427ab8cdf20f0a516292328dcb1cc3ba3790ad467c49a6faa548aae0e: Status 404 returned error can't find the container with id da6d778427ab8cdf20f0a516292328dcb1cc3ba3790ad467c49a6faa548aae0e
Sep 10 21:15:13.026274 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:13.026239    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7" event={"ID":"66b1c2bb-15b5-4933-8dd4-4a7406ace11e","Type":"ContainerStarted","Data":"7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701"}
Sep 10 21:15:13.026517 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:13.026292    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7" event={"ID":"66b1c2bb-15b5-4933-8dd4-4a7406ace11e","Type":"ContainerStarted","Data":"da6d778427ab8cdf20f0a516292328dcb1cc3ba3790ad467c49a6faa548aae0e"}
Sep 10 21:15:13.026517 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:13.026435    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:13.028018 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:13.027990    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt" event={"ID":"fb022a71-1c63-4622-af8a-2bd0a42eddec","Type":"ContainerStarted","Data":"0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5"}
Sep 10 21:15:13.028141 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:13.028125    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:13.045934 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:13.045814    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7" podStartSLOduration=1.045795677 podStartE2EDuration="1.045795677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:13.044595887 +0000 UTC m=+1007.370356807" watchObservedRunningTime="2026-09-10 21:15:13.045795677 +0000 UTC m=+1007.371556599"
Sep 10 21:15:13.069116 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:13.069047    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt" podStartSLOduration=1.7468460270000001 podStartE2EDuration="2.069031631s" totalImagesPullingTime="322.185604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:13.066638501 +0000 UTC m=+1007.392399422" watchObservedRunningTime="2026-09-10 21:15:13.069031631 +0000 UTC m=+1007.394792551"
Sep 10 21:15:15.318733 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:15.318696    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:15.420895 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:15.420858    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:15.572744 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:15.572639    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:15.924255 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:15.924211    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:16.025397 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:16.025354    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:16.128608 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:16.128568    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:19.039340 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:19.039304    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:15:22.066808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:22.066763    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:22.146940 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:22.146903    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:22.236351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:22.236306    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:22.336775 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:22.336680    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:22.436457 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:22.436401    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:24.035895 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:24.035857    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:15:24.577598 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:24.577550    2584 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:59954->10.132.0.55:15021: read: connection reset by peer" start-of-body=
Sep 10 21:15:24.577790 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:24.577639    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:59954->10.132.0.55:15021: read: connection reset by peer"
Sep 10 21:15:25.101307 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:25.101272    2584 generic.go:356] "Generic (PLEG): container finished" podID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerID="a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0" exitCode=0
Sep 10 21:15:25.101721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:25.101346    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" event={"ID":"e7fdb282-b518-41ab-b67a-75bf7c41ec27","Type":"ContainerDied","Data":"a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0"}
Sep 10 21:15:25.101791 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:25.101777    2584 scope.go:117] "RemoveContainer" containerID="a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0"
Sep 10 21:15:25.103767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:25.103735    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:15:25.103874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:25.103806    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:15:25.103874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:25.103839    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 10 21:15:26.109253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:26.109212    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" event={"ID":"e7fdb282-b518-41ab-b67a-75bf7c41ec27","Type":"ContainerStarted","Data":"014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3"}
Sep 10 21:15:26.834796 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:26.834754    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:26.834982 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:26.834940    2584 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:15:26.835039 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:26.834999    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 21:15:27.835097 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:27.835062    2584 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:15:27.835531 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:27.835129    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 21:15:28.654556 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:28.654521    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:28.781490 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:28.781452    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:28.835153 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:28.835119    2584 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:15:28.835604 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:28.835185    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 21:15:29.536669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:29.536628    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:29.835021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:29.834928    2584 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:15:29.835021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:29.834987    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 21:15:29.947565 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:29.947533    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:30.224081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:30.224046    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:30.834989 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:30.834949    2584 patch_prober.go:28] interesting pod/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:15:30.835173 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:30.835019    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 21:15:31.839166 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:31.839129    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:31.839695 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:31.839435    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:31.840200 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:31.840176    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:15:33.040859 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:33.040824    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:33.150645 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:33.150606    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:34.846433 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.846384    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"]
Sep 10 21:15:34.853586 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.853557    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.855890 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.855865    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-290e03-serving-cert\""
Sep 10 21:15:34.862232 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.861551    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"]
Sep 10 21:15:34.874683 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.874651    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.874819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.874705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwh9n\" (UniqueName: \"kubernetes.io/projected/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-kube-api-access-xwh9n\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.874819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.874789    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.874907 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.874838    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.885489 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.885462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:34.975601 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.975561    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.975789 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.975680    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.975846 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.975785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.975846 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.975830    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwh9n\" (UniqueName: \"kubernetes.io/projected/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-kube-api-access-xwh9n\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.975945 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:15:34.975876    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-serving-cert: secret "maas-api-e2e-shared-a-w2-290e03-serving-cert" not found
Sep 10 21:15:34.976001 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:15:34.975945    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls podName:91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2 nodeName:}" failed. No retries permitted until 2026-09-10 21:15:35.475922271 +0000 UTC m=+1029.801683171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls") pod "maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg" (UID: "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2") : secret "maas-api-e2e-shared-a-w2-290e03-serving-cert" not found
Sep 10 21:15:34.976303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.976276    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.978327 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.978301    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.985491 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.985465    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwh9n\" (UniqueName: \"kubernetes.io/projected/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-kube-api-access-xwh9n\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:34.986279 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:34.986242    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.162153 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.162117    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.264675 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.264643    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.481774 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.481690    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:35.484326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.484293    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:35.491819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.491785    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.603742 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.603705    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.766161 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.766068    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:35.877940 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.877837    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.981754 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:35.981722    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:36.046361 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:36.046266    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"]
Sep 10 21:15:36.054524 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:36.054380    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:36.160837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:36.160795    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg" event={"ID":"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2","Type":"ContainerStarted","Data":"3a186ec00478eee06b2b61872089873fc6bef4fdda9cbe261ab95f041b89121e"}
Sep 10 21:15:36.841714 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:36.841628    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:37.167793 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:37.167758    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg" event={"ID":"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2","Type":"ContainerStarted","Data":"da63e356965f140e0c9ab452a295b7cb184daff377308b96b11d8fb685119df5"}
Sep 10 21:15:37.168199 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:37.167870    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:37.188523 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:37.188469    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg" podStartSLOduration=2.721566561 podStartE2EDuration="3.188452489s" totalImagesPullingTime="466.885928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:37.186218969 +0000 UTC m=+1031.511979894" watchObservedRunningTime="2026-09-10 21:15:37.188452489 +0000 UTC m=+1031.514213412"
Sep 10 21:15:39.061555 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:39.061522    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:39.155413 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:39.155378    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:40.193881 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.193847    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"]
Sep 10 21:15:40.196595 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.196571    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.198720 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.198698    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-755926-serving-cert\""
Sep 10 21:15:40.207553 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.207526    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"]
Sep 10 21:15:40.226397 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.226369    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88f4j\" (UniqueName: \"kubernetes.io/projected/62785e98-9236-4751-b0c0-488a7bf5ca79-kube-api-access-88f4j\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.226548 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.226405    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.226548 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.226511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62785e98-9236-4751-b0c0-488a7bf5ca79-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.226548 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.226539    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.244365 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.244334    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:40.327685 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.327647    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88f4j\" (UniqueName: \"kubernetes.io/projected/62785e98-9236-4751-b0c0-488a7bf5ca79-kube-api-access-88f4j\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.327836 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.327702    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.327836 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.327765    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62785e98-9236-4751-b0c0-488a7bf5ca79-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.327836 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.327798    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.328032 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:15:40.328005    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-serving-cert: secret "maas-api-e2e-shared-b-w3-755926-serving-cert" not found
Sep 10 21:15:40.328110 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:15:40.328099    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls podName:62785e98-9236-4751-b0c0-488a7bf5ca79 nodeName:}" failed. No retries permitted until 2026-09-10 21:15:40.828082053 +0000 UTC m=+1035.153842957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls") pod "maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n" (UID: "62785e98-9236-4751-b0c0-488a7bf5ca79") : secret "maas-api-e2e-shared-b-w3-755926-serving-cert" not found
Sep 10 21:15:40.328534 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.328511    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62785e98-9236-4751-b0c0-488a7bf5ca79-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.330205 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.330180    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.335890 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.335865    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:40.338173 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.338152    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88f4j\" (UniqueName: \"kubernetes.io/projected/62785e98-9236-4751-b0c0-488a7bf5ca79-kube-api-access-88f4j\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.436364 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.436328    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:40.796116 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.796073    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:40.832253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.832222    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.834583 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.834565    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:40.897407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.897365    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:40.903702 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.903672    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"]
Sep 10 21:15:40.906326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.906309    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:40.911323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.911299    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-755926\""
Sep 10 21:15:40.920185 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.920161    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"]
Sep 10 21:15:40.933610 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.933580    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41dcd303-fd05-480f-be8e-7de40e96f5b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:40.933746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:40.933714    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb79j\" (UniqueName: \"kubernetes.io/projected/41dcd303-fd05-480f-be8e-7de40e96f5b8-kube-api-access-zb79j\") pod \"payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:41.035253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.035178    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb79j\" (UniqueName: \"kubernetes.io/projected/41dcd303-fd05-480f-be8e-7de40e96f5b8-kube-api-access-zb79j\") pod \"payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:41.035486 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.035339    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41dcd303-fd05-480f-be8e-7de40e96f5b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:41.036329 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.036299    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41dcd303-fd05-480f-be8e-7de40e96f5b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:41.044581 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.044555    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb79j\" (UniqueName: \"kubernetes.io/projected/41dcd303-fd05-480f-be8e-7de40e96f5b8-kube-api-access-zb79j\") pod \"payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:41.108212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.108129    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:41.217538 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.217500    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:41.382162 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.382134    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"]
Sep 10 21:15:41.384698 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:15:41.384671    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62785e98_9236_4751_b0c0_488a7bf5ca79.slice/crio-4aaa68da95d4663f27ae7e22d3caf56c769dacc6f60b3b1990a8a2bc1a06b820 WatchSource:0}: Error finding container 4aaa68da95d4663f27ae7e22d3caf56c769dacc6f60b3b1990a8a2bc1a06b820: Status 404 returned error can't find the container with id 4aaa68da95d4663f27ae7e22d3caf56c769dacc6f60b3b1990a8a2bc1a06b820
Sep 10 21:15:41.489649 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:41.489622    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"]
Sep 10 21:15:41.492154 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:15:41.492117    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41dcd303_fd05_480f_be8e_7de40e96f5b8.slice/crio-86412ea22177852a5dcab00f518d57d0e28a52321b2c06547582f48f0a56997a WatchSource:0}: Error finding container 86412ea22177852a5dcab00f518d57d0e28a52321b2c06547582f48f0a56997a: Status 404 returned error can't find the container with id 86412ea22177852a5dcab00f518d57d0e28a52321b2c06547582f48f0a56997a
Sep 10 21:15:42.197704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.197663    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n" event={"ID":"62785e98-9236-4751-b0c0-488a7bf5ca79","Type":"ContainerStarted","Data":"3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347"}
Sep 10 21:15:42.197876 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.197738    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:42.197876 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.197762    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n" event={"ID":"62785e98-9236-4751-b0c0-488a7bf5ca79","Type":"ContainerStarted","Data":"4aaa68da95d4663f27ae7e22d3caf56c769dacc6f60b3b1990a8a2bc1a06b820"}
Sep 10 21:15:42.199236 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.199208    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5" event={"ID":"41dcd303-fd05-480f-be8e-7de40e96f5b8","Type":"ContainerStarted","Data":"0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac"}
Sep 10 21:15:42.199364 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.199266    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:42.199364 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.199284    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5" event={"ID":"41dcd303-fd05-480f-be8e-7de40e96f5b8","Type":"ContainerStarted","Data":"86412ea22177852a5dcab00f518d57d0e28a52321b2c06547582f48f0a56997a"}
Sep 10 21:15:42.217455 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.217394    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n" podStartSLOduration=1.837489992 podStartE2EDuration="2.217376679s" totalImagesPullingTime="379.886687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:42.215711996 +0000 UTC m=+1036.541472917" watchObservedRunningTime="2026-09-10 21:15:42.217376679 +0000 UTC m=+1036.543137600"
Sep 10 21:15:42.233056 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:42.233008    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5" podStartSLOduration=2.232993242 podStartE2EDuration="2.232993242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:42.232079134 +0000 UTC m=+1036.557840056" watchObservedRunningTime="2026-09-10 21:15:42.232993242 +0000 UTC m=+1036.558754166"
Sep 10 21:15:43.178865 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:43.178832    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:15:48.209596 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:48.209563    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:15:53.206573 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:53.206532    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:15:56.076515 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:56.076472    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:57.903560 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:57.903524    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:58.225975 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:58.225877    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:58.564196 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:58.564108    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:58.950430 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:15:58.950396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:01.196763 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:01.196716    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:01.283382 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:01.283345    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:01.347447 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:01.347383    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:02.919589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:02.919547    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:03.020137 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.020105    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:03.277698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.277662    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:03.437365 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.437325    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:03.536476 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.536365    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:03.567537 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.567502    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"]
Sep 10 21:16:03.570544 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.570517    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.573356 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.573132    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-9f71ac\""
Sep 10 21:16:03.573356 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.573147    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-9f71ac-dockercfg-wxlx7\""
Sep 10 21:16:03.582530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.582503    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"]
Sep 10 21:16:03.625640 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.625606    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"]
Sep 10 21:16:03.628156 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.628136    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.639335 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.639307    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"]
Sep 10 21:16:03.643271 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.643241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p88p6\" (UniqueName: \"kubernetes.io/projected/9b0b36db-32ec-4618-8652-d44a1761655d-kube-api-access-p88p6\") pod \"payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.643368 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.643343    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b0b36db-32ec-4618-8652-d44a1761655d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.744380 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.744335    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b0b36db-32ec-4618-8652-d44a1761655d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.744589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.744412    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7mnm\" (UniqueName: \"kubernetes.io/projected/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-kube-api-access-k7mnm\") pod \"payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.744589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.744571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p88p6\" (UniqueName: \"kubernetes.io/projected/9b0b36db-32ec-4618-8652-d44a1761655d-kube-api-access-p88p6\") pod \"payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.744683 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.744608    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.745309 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.745285    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b0b36db-32ec-4618-8652-d44a1761655d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.753276 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.753242    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p88p6\" (UniqueName: \"kubernetes.io/projected/9b0b36db-32ec-4618-8652-d44a1761655d-kube-api-access-p88p6\") pod \"payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.845609 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.845507    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7mnm\" (UniqueName: \"kubernetes.io/projected/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-kube-api-access-k7mnm\") pod \"payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.845785 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.845640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.846368 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.846342    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.855721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.855682    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7mnm\" (UniqueName: \"kubernetes.io/projected/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-kube-api-access-k7mnm\") pod \"payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.883482 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.883434    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:03.884619 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.884587    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:03.939261 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.939193    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:03.996746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:03.996690    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:04.156617 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.156585    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"]
Sep 10 21:16:04.159148 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:16:04.159120    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0b36db_32ec_4618_8652_d44a1761655d.slice/crio-b850b93bdd4ef5a649c72ceb0e7f335f4b745ff78bad1faf6f0af57325163fa8 WatchSource:0}: Error finding container b850b93bdd4ef5a649c72ceb0e7f335f4b745ff78bad1faf6f0af57325163fa8: Status 404 returned error can't find the container with id b850b93bdd4ef5a649c72ceb0e7f335f4b745ff78bad1faf6f0af57325163fa8
Sep 10 21:16:04.226319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.226286    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"]
Sep 10 21:16:04.228802 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:16:04.228764    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9e2e13_3716_4ce9_b682_cc77fc9b602e.slice/crio-48c7f5e4871d955830ef0a03bcca9e6103c99da1ac39cc4cb8c59e6ffb078274 WatchSource:0}: Error finding container 48c7f5e4871d955830ef0a03bcca9e6103c99da1ac39cc4cb8c59e6ffb078274: Status 404 returned error can't find the container with id 48c7f5e4871d955830ef0a03bcca9e6103c99da1ac39cc4cb8c59e6ffb078274
Sep 10 21:16:04.322888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.322527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x" event={"ID":"2a9e2e13-3716-4ce9-b682-cc77fc9b602e","Type":"ContainerStarted","Data":"077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a"}
Sep 10 21:16:04.322888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.322593    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x" event={"ID":"2a9e2e13-3716-4ce9-b682-cc77fc9b602e","Type":"ContainerStarted","Data":"48c7f5e4871d955830ef0a03bcca9e6103c99da1ac39cc4cb8c59e6ffb078274"}
Sep 10 21:16:04.322888 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.322643    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:04.324444 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.324395    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk" event={"ID":"9b0b36db-32ec-4618-8652-d44a1761655d","Type":"ContainerStarted","Data":"2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552"}
Sep 10 21:16:04.324571 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.324464    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:04.324571 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.324481    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk" event={"ID":"9b0b36db-32ec-4618-8652-d44a1761655d","Type":"ContainerStarted","Data":"b850b93bdd4ef5a649c72ceb0e7f335f4b745ff78bad1faf6f0af57325163fa8"}
Sep 10 21:16:04.339986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.339936    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x" podStartSLOduration=1.3399204679999999 podStartE2EDuration="1.339920468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:04.338154635 +0000 UTC m=+1058.663915558" watchObservedRunningTime="2026-09-10 21:16:04.339920468 +0000 UTC m=+1058.665681391"
Sep 10 21:16:04.356480 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.356412    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk" podStartSLOduration=1.35639701 podStartE2EDuration="1.35639701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:04.353627131 +0000 UTC m=+1058.679388055" watchObservedRunningTime="2026-09-10 21:16:04.35639701 +0000 UTC m=+1058.682157931"
Sep 10 21:16:04.809658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:04.809622    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:09.656560 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:09.656523    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:15.331831 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:15.331798    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:16:15.332226 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:15.332119    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:16:26.121132 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:26.121051    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:28.878496 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:28.878461    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:35.616506 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:35.616461    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:35.955365 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:35.955313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:36.273889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:36.273793    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.372605 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:39.372566    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.484228 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:39.484188    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:40.495256 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.495216    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"]
Sep 10 21:16:40.499154 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.499129    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.501723 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.501699    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8b5fd067-serving-cert\""
Sep 10 21:16:40.510113 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.510087    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"]
Sep 10 21:16:40.542766 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.542737    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:40.596587 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.596541    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.596765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.596662    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-metrics-certs\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.596765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.596704    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smfmq\" (UniqueName: \"kubernetes.io/projected/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-kube-api-access-smfmq\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.596765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.596744    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-openshift-service-ca\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.649767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.649713    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:40.698077 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.698035    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-metrics-certs\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.698263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.698087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smfmq\" (UniqueName: \"kubernetes.io/projected/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-kube-api-access-smfmq\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.698263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.698132    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-openshift-service-ca\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.698263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.698218    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.698485 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:16:40.698367    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-serving-cert: secret "maas-api-e2e-mt-8b5fd067-serving-cert" not found
Sep 10 21:16:40.698485 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:16:40.698472    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls podName:b730fdc0-80fd-4be1-9898-37f3e9f5b6ab nodeName:}" failed. No retries permitted until 2026-09-10 21:16:41.19845099 +0000 UTC m=+1095.524211893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls") pod "maas-api-e2e-mt-8b5fd067-dfd754697-sh58n" (UID: "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab") : secret "maas-api-e2e-mt-8b5fd067-serving-cert" not found
Sep 10 21:16:40.698860 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.698839    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-openshift-service-ca\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.700514 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.700487    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-metrics-certs\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.709098 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.709074    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smfmq\" (UniqueName: \"kubernetes.io/projected/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-kube-api-access-smfmq\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:40.767871 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:40.767785    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:41.169082 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.169052    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:41.202264 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.202231    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:41.204811 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.204784    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls\") pod \"maas-api-e2e-mt-8b5fd067-dfd754697-sh58n\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:41.272587 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.272545    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:41.301299 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.301266    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"]
Sep 10 21:16:41.306157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.306123    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.313110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.312210    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8b5fd067-dockercfg-djf5v\""
Sep 10 21:16:41.313110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.312352    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8b5fd067\""
Sep 10 21:16:41.317508 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.317481    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"]
Sep 10 21:16:41.405913 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.405873    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36eebb53-355c-4d17-a9ce-a79083605f9e-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") " pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.406135 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.406012    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2dbf\" (UniqueName: \"kubernetes.io/projected/36eebb53-355c-4d17-a9ce-a79083605f9e-kube-api-access-c2dbf\") pod \"payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") " pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.412687 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.412661    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:41.507321 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.507219    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36eebb53-355c-4d17-a9ce-a79083605f9e-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") " pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.507835 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.507451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2dbf\" (UniqueName: \"kubernetes.io/projected/36eebb53-355c-4d17-a9ce-a79083605f9e-kube-api-access-c2dbf\") pod \"payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") " pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.508230 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.508205    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36eebb53-355c-4d17-a9ce-a79083605f9e-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") " pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.516294 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.516268    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2dbf\" (UniqueName: \"kubernetes.io/projected/36eebb53-355c-4d17-a9ce-a79083605f9e-kube-api-access-c2dbf\") pod \"payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") " pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.623871 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.623829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:41.694489 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:16:41.694388    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb730fdc0_80fd_4be1_9898_37f3e9f5b6ab.slice/crio-a8aecbfff6ead9d665887f1699337e56a0394aa498c8f79a990710629ed554bb WatchSource:0}: Error finding container a8aecbfff6ead9d665887f1699337e56a0394aa498c8f79a990710629ed554bb: Status 404 returned error can't find the container with id a8aecbfff6ead9d665887f1699337e56a0394aa498c8f79a990710629ed554bb
Sep 10 21:16:41.694663 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.694640    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"]
Sep 10 21:16:41.903369 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:41.903337    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"]
Sep 10 21:16:41.904723 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:16:41.904696    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36eebb53_355c_4d17_a9ce_a79083605f9e.slice/crio-1c709d2345e513209bad46eb239160dadcfb6bd86d4dab0f622d01ead759d775 WatchSource:0}: Error finding container 1c709d2345e513209bad46eb239160dadcfb6bd86d4dab0f622d01ead759d775: Status 404 returned error can't find the container with id 1c709d2345e513209bad46eb239160dadcfb6bd86d4dab0f622d01ead759d775
Sep 10 21:16:42.528831 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.528758    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr" event={"ID":"36eebb53-355c-4d17-a9ce-a79083605f9e","Type":"ContainerStarted","Data":"fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63"}
Sep 10 21:16:42.528831 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.528826    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr" event={"ID":"36eebb53-355c-4d17-a9ce-a79083605f9e","Type":"ContainerStarted","Data":"1c709d2345e513209bad46eb239160dadcfb6bd86d4dab0f622d01ead759d775"}
Sep 10 21:16:42.529306 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.528945    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:42.530461 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.530435    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n" event={"ID":"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab","Type":"ContainerStarted","Data":"1192c34df9ec9198ba6382ac26c59967e61cd2a611d8426a51c5f655aada1686"}
Sep 10 21:16:42.530568 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.530487    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:42.530568 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.530512    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n" event={"ID":"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab","Type":"ContainerStarted","Data":"a8aecbfff6ead9d665887f1699337e56a0394aa498c8f79a990710629ed554bb"}
Sep 10 21:16:42.549707 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.549657    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr" podStartSLOduration=1.549638842 podStartE2EDuration="1.549638842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:42.547266351 +0000 UTC m=+1096.873027274" watchObservedRunningTime="2026-09-10 21:16:42.549638842 +0000 UTC m=+1096.875399766"
Sep 10 21:16:42.565524 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:42.565473    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n" podStartSLOduration=2.179070726 podStartE2EDuration="2.565462149s" totalImagesPullingTime="386.391423ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:42.563611239 +0000 UTC m=+1096.889372159" watchObservedRunningTime="2026-09-10 21:16:42.565462149 +0000 UTC m=+1096.891223070"
Sep 10 21:16:48.541653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:48.541621    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:16:49.748623 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:49.748592    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:49.850598 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:49.850560    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:53.538359 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:53.538325    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:16:57.815653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:16:57.815614    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:04.299833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:04.299790    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:09.158329 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:09.158295    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:15.095224 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:15.095185    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:48.753909 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.753816    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"]
Sep 10 21:17:48.754543 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.754093    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n" podUID="b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" containerName="maas-api" containerID="cri-o://1192c34df9ec9198ba6382ac26c59967e61cd2a611d8426a51c5f655aada1686" gracePeriod=30
Sep 10 21:17:48.788928 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.788892    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:48.823338 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.823305    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"]
Sep 10 21:17:48.823618 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.823590    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr" podUID="36eebb53-355c-4d17-a9ce-a79083605f9e" containerName="payload-processing" containerID="cri-o://fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63" gracePeriod=30
Sep 10 21:17:48.890932 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.890893    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:48.893319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.893291    2584 generic.go:356] "Generic (PLEG): container finished" podID="b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" containerID="1192c34df9ec9198ba6382ac26c59967e61cd2a611d8426a51c5f655aada1686" exitCode=0
Sep 10 21:17:48.893471 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.893379    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n" event={"ID":"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab","Type":"ContainerDied","Data":"1192c34df9ec9198ba6382ac26c59967e61cd2a611d8426a51c5f655aada1686"}
Sep 10 21:17:48.992685 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:48.992644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:49.093738 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.093651    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:49.118001 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.117976    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:17:49.121502 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.121483    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:17:49.174962 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.174933    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smfmq\" (UniqueName: \"kubernetes.io/projected/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-kube-api-access-smfmq\") pod \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") "
Sep 10 21:17:49.175126 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175011    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-openshift-service-ca\") pod \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") "
Sep 10 21:17:49.175126 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175052    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-metrics-certs\") pod \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") "
Sep 10 21:17:49.175126 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175084    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36eebb53-355c-4d17-a9ce-a79083605f9e-plugins-config-volume\") pod \"36eebb53-355c-4d17-a9ce-a79083605f9e\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") "
Sep 10 21:17:49.175480 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175306    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2dbf\" (UniqueName: \"kubernetes.io/projected/36eebb53-355c-4d17-a9ce-a79083605f9e-kube-api-access-c2dbf\") pod \"36eebb53-355c-4d17-a9ce-a79083605f9e\" (UID: \"36eebb53-355c-4d17-a9ce-a79083605f9e\") "
Sep 10 21:17:49.175480 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175369    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls\") pod \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\" (UID: \"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab\") "
Sep 10 21:17:49.175480 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175456    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" (UID: "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:49.175660 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175473    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36eebb53-355c-4d17-a9ce-a79083605f9e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36eebb53-355c-4d17-a9ce-a79083605f9e" (UID: "36eebb53-355c-4d17-a9ce-a79083605f9e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:49.175812 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175785    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.175900 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.175816    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36eebb53-355c-4d17-a9ce-a79083605f9e-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.177783 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.177754    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" (UID: "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:17:49.178059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.178023    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-kube-api-access-smfmq" (OuterVolumeSpecName: "kube-api-access-smfmq") pod "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" (UID: "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab"). InnerVolumeSpecName "kube-api-access-smfmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:49.178169 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.178071    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" (UID: "b730fdc0-80fd-4be1-9898-37f3e9f5b6ab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:17:49.178169 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.178084    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36eebb53-355c-4d17-a9ce-a79083605f9e-kube-api-access-c2dbf" (OuterVolumeSpecName: "kube-api-access-c2dbf") pod "36eebb53-355c-4d17-a9ce-a79083605f9e" (UID: "36eebb53-355c-4d17-a9ce-a79083605f9e"). InnerVolumeSpecName "kube-api-access-c2dbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:49.277206 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.277172    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2dbf\" (UniqueName: \"kubernetes.io/projected/36eebb53-355c-4d17-a9ce-a79083605f9e-kube-api-access-c2dbf\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.277206 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.277203    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.277404 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.277239    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smfmq\" (UniqueName: \"kubernetes.io/projected/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-kube-api-access-smfmq\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.277404 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.277249    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.899638 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.899603    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"
Sep 10 21:17:49.900104 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.899638    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n" event={"ID":"b730fdc0-80fd-4be1-9898-37f3e9f5b6ab","Type":"ContainerDied","Data":"a8aecbfff6ead9d665887f1699337e56a0394aa498c8f79a990710629ed554bb"}
Sep 10 21:17:49.900104 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.899706    2584 scope.go:117] "RemoveContainer" containerID="1192c34df9ec9198ba6382ac26c59967e61cd2a611d8426a51c5f655aada1686"
Sep 10 21:17:49.901179 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.901031    2584 generic.go:356] "Generic (PLEG): container finished" podID="36eebb53-355c-4d17-a9ce-a79083605f9e" containerID="fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63" exitCode=0
Sep 10 21:17:49.901179 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.901069    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"
Sep 10 21:17:49.901179 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.901112    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr" event={"ID":"36eebb53-355c-4d17-a9ce-a79083605f9e","Type":"ContainerDied","Data":"fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63"}
Sep 10 21:17:49.901179 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.901168    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr" event={"ID":"36eebb53-355c-4d17-a9ce-a79083605f9e","Type":"ContainerDied","Data":"1c709d2345e513209bad46eb239160dadcfb6bd86d4dab0f622d01ead759d775"}
Sep 10 21:17:49.910282 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.910268    2584 scope.go:117] "RemoveContainer" containerID="fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63"
Sep 10 21:17:49.918382 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.918362    2584 scope.go:117] "RemoveContainer" containerID="fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63"
Sep 10 21:17:49.918687 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:17:49.918666    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63\": container with ID starting with fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63 not found: ID does not exist" containerID="fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63"
Sep 10 21:17:49.918741 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.918698    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63"} err="failed to get container status \"fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63\": rpc error: code = NotFound desc = could not find container \"fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63\": container with ID starting with fba211be940ffd852f37bb4c91bd62d02e39cbc1143c33d2a217daa51787dd63 not found: ID does not exist"
Sep 10 21:17:49.923715 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.923691    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"]
Sep 10 21:17:49.927633 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.927610    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b5fd067-dfd754697-sh58n"]
Sep 10 21:17:49.938298 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.938275    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"]
Sep 10 21:17:49.942236 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:49.942213    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b5fd067-5f9f575d57-2wtjr"]
Sep 10 21:17:50.301180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:50.301147    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36eebb53-355c-4d17-a9ce-a79083605f9e" path="/var/lib/kubelet/pods/36eebb53-355c-4d17-a9ce-a79083605f9e/volumes"
Sep 10 21:17:50.301585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:50.301571    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" path="/var/lib/kubelet/pods/b730fdc0-80fd-4be1-9898-37f3e9f5b6ab/volumes"
Sep 10 21:17:52.136559 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:52.136508    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:56.695953 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:56.695915    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:57.174045 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:17:57.173995    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:18.820733 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:18.820696    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:26.193603 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:26.193565    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:18:26.194750 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:26.194722    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:18:26.304821 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:26.304793    2584 scope.go:117] "RemoveContainer" containerID="14103acad3cd52fa3b7ff85d30c2edcc0c6e58a65d9bdc327443336e3402c993"
Sep 10 21:18:33.484565 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:33.484519    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:34.621428 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:34.621389    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:40.812475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:40.812437    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:46.823481 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:46.823441    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:47.881006 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:47.880966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:47.986798 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:47.986765    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:48.172131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:48.172096    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:48.905528 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:48.905494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:49.416927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:49.416887    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:49.515434 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:49.515393    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:49.625176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:49.625148    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:50.199604 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:50.199568    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:50.298462 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:50.298398    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:50.398381 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:50.398345    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:51.655936 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:51.655896    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:58.205542 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:58.205507    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:58.306507 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:58.306472    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:58.863548 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:18:58.863508    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:02.773807 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:02.773767    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:05.544318 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:05.544284    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:05.615240 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:05.615203    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:05.701145 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:05.701113    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:11.754701 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:11.754665    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:16.259178 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:16.259127    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:16.482659 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:16.482629    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:32.891745 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:32.891656    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:37.489935 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:37.489905    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:41.535493 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:41.535453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:41.666819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:41.666786    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:45.369478 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:45.369438    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:45.501736 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:45.501680    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:45.809582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:45.809551    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:46.348100 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:46.348063    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:46.621061 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:46.620966    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:49.611828 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:49.611794    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:49.811503 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:49.811468    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:50.936028 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.935989    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"]
Sep 10 21:19:50.936653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.936623    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" containerName="maas-api"
Sep 10 21:19:50.936653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.936645    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" containerName="maas-api"
Sep 10 21:19:50.936833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.936659    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36eebb53-355c-4d17-a9ce-a79083605f9e" containerName="payload-processing"
Sep 10 21:19:50.936833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.936667    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="36eebb53-355c-4d17-a9ce-a79083605f9e" containerName="payload-processing"
Sep 10 21:19:50.936833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.936759    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="36eebb53-355c-4d17-a9ce-a79083605f9e" containerName="payload-processing"
Sep 10 21:19:50.936833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.936774    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b730fdc0-80fd-4be1-9898-37f3e9f5b6ab" containerName="maas-api"
Sep 10 21:19:50.940001 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.939984    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:50.950288 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:50.950266    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5a30bd-serving-cert\""
Sep 10 21:19:51.010119 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.010085    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"]
Sep 10 21:19:51.034398 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.034364    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:51.039472 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.039444    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s27db\" (UniqueName: \"kubernetes.io/projected/5d622e70-afd8-4b3c-a2d2-353d3a678726-kube-api-access-s27db\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.039594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.039496    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d622e70-afd8-4b3c-a2d2-353d3a678726-openshift-service-ca\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.039594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.039528    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-metrics-certs\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.039594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.039569    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.140123 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.140096    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.140280 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.140180    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s27db\" (UniqueName: \"kubernetes.io/projected/5d622e70-afd8-4b3c-a2d2-353d3a678726-kube-api-access-s27db\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.140280 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.140209    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d622e70-afd8-4b3c-a2d2-353d3a678726-openshift-service-ca\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.140280 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:19:51.140244    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-serving-cert: secret "maas-api-e2e-api-a-5a30bd-serving-cert" not found
Sep 10 21:19:51.140464 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:19:51.140314    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls podName:5d622e70-afd8-4b3c-a2d2-353d3a678726 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:51.640295255 +0000 UTC m=+1285.966056160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls") pod "maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv" (UID: "5d622e70-afd8-4b3c-a2d2-353d3a678726") : secret "maas-api-e2e-api-a-5a30bd-serving-cert" not found
Sep 10 21:19:51.140464 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.140250    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-metrics-certs\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.140938 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.140916    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d622e70-afd8-4b3c-a2d2-353d3a678726-openshift-service-ca\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.142614 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.142598    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-metrics-certs\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.159243 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.159221    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s27db\" (UniqueName: \"kubernetes.io/projected/5d622e70-afd8-4b3c-a2d2-353d3a678726-kube-api-access-s27db\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.264284 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.264195    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:51.402137 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.402103    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:51.646344 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.646317    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.648851 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.648829    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls\") pod \"maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.851659 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.851614    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:51.890668 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:51.890634    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:52.009340 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.009307    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:52.161837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.161804    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"]
Sep 10 21:19:52.164402 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:19:52.164356    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d622e70_afd8_4b3c_a2d2_353d3a678726.slice/crio-1b6173b1e2375adca7eaf4668861b84c20084234153bcfedf99386f0d50f0603 WatchSource:0}: Error finding container 1b6173b1e2375adca7eaf4668861b84c20084234153bcfedf99386f0d50f0603: Status 404 returned error can't find the container with id 1b6173b1e2375adca7eaf4668861b84c20084234153bcfedf99386f0d50f0603
Sep 10 21:19:52.376795 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.376760    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"]
Sep 10 21:19:52.386625 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.386594    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.396629 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.396598    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"]
Sep 10 21:19:52.398534 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.398506    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-5a30bd-dockercfg-swc79\""
Sep 10 21:19:52.398827 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.398806    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-5a30bd\""
Sep 10 21:19:52.454444 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.454328    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50213540-3b93-46bf-b2cc-04baacc5e175-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") " pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.454444 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.454399    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stmdb\" (UniqueName: \"kubernetes.io/projected/50213540-3b93-46bf-b2cc-04baacc5e175-kube-api-access-stmdb\") pod \"payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") " pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.555401 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.555370    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stmdb\" (UniqueName: \"kubernetes.io/projected/50213540-3b93-46bf-b2cc-04baacc5e175-kube-api-access-stmdb\") pod \"payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") " pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.555581 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.555496    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50213540-3b93-46bf-b2cc-04baacc5e175-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") " pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.556127 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.556109    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50213540-3b93-46bf-b2cc-04baacc5e175-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") " pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.567052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.567022    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv" event={"ID":"5d622e70-afd8-4b3c-a2d2-353d3a678726","Type":"ContainerStarted","Data":"1b6173b1e2375adca7eaf4668861b84c20084234153bcfedf99386f0d50f0603"}
Sep 10 21:19:52.568820 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.568796    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stmdb\" (UniqueName: \"kubernetes.io/projected/50213540-3b93-46bf-b2cc-04baacc5e175-kube-api-access-stmdb\") pod \"payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") " pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.699683 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.699648    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:52.982725 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:52.982702    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"]
Sep 10 21:19:52.985365 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:19:52.985331    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50213540_3b93_46bf_b2cc_04baacc5e175.slice/crio-f47e51a894ee8ef279415c5e178541f496c085e3d290c78df1a5d3a70977652d WatchSource:0}: Error finding container f47e51a894ee8ef279415c5e178541f496c085e3d290c78df1a5d3a70977652d: Status 404 returned error can't find the container with id f47e51a894ee8ef279415c5e178541f496c085e3d290c78df1a5d3a70977652d
Sep 10 21:19:53.575866 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:53.575814    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv" event={"ID":"5d622e70-afd8-4b3c-a2d2-353d3a678726","Type":"ContainerStarted","Data":"06e2209d93641ba154b8e1ee015454bde7e316a8f4fc3318abecb8a777b5ad1c"}
Sep 10 21:19:53.576360 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:53.576025    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:19:53.577619 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:53.577591    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4" event={"ID":"50213540-3b93-46bf-b2cc-04baacc5e175","Type":"ContainerStarted","Data":"ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8"}
Sep 10 21:19:53.577758 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:53.577640    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4" event={"ID":"50213540-3b93-46bf-b2cc-04baacc5e175","Type":"ContainerStarted","Data":"f47e51a894ee8ef279415c5e178541f496c085e3d290c78df1a5d3a70977652d"}
Sep 10 21:19:53.577758 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:53.577669    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:19:53.603562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:53.603502    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv" podStartSLOduration=3.200913633 podStartE2EDuration="3.6034874s" totalImagesPullingTime="402.573767ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:53.600030036 +0000 UTC m=+1287.925790958" watchObservedRunningTime="2026-09-10 21:19:53.6034874 +0000 UTC m=+1287.929248320"
Sep 10 21:19:53.626187 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:53.626139    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4" podStartSLOduration=1.626124394 podStartE2EDuration="1.626124394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:53.624610325 +0000 UTC m=+1287.950371248" watchObservedRunningTime="2026-09-10 21:19:53.626124394 +0000 UTC m=+1287.951885346"
Sep 10 21:19:59.589182 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:19:59.589149    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:20:04.586836 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:04.586793    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:20:12.163622 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:12.163586    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:12.261831 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:12.261787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:14.525680 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:14.525644    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:14.985137 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:14.985077    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:15.403499 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:15.403462    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:15.602433 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:15.602382    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:18.773312 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:18.773267    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:18.920937 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:18.920900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.095219 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.095183    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"]
Sep 10 21:20:20.100341 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.100317    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.103830 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.103805    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a22115-serving-cert\""
Sep 10 21:20:20.115362 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.115333    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"]
Sep 10 21:20:20.129758 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.129725    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxrmq\" (UniqueName: \"kubernetes.io/projected/93e299c3-4337-4b31-a066-75dea0278b7f-kube-api-access-sxrmq\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.129891 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.129792    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-maas-api-tls\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.129960 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.129893    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e299c3-4337-4b31-a066-75dea0278b7f-openshift-service-ca\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.129960 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.129945    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-metrics-certs\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.192619 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.192592    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.230537 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.230503    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxrmq\" (UniqueName: \"kubernetes.io/projected/93e299c3-4337-4b31-a066-75dea0278b7f-kube-api-access-sxrmq\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.230700 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.230560    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-maas-api-tls\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.230700 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.230600    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e299c3-4337-4b31-a066-75dea0278b7f-openshift-service-ca\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.230700 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.230623    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-metrics-certs\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.231326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.231300    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e299c3-4337-4b31-a066-75dea0278b7f-openshift-service-ca\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.233052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.233030    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-metrics-certs\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.233176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.233083    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-maas-api-tls\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.241909 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.241888    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxrmq\" (UniqueName: \"kubernetes.io/projected/93e299c3-4337-4b31-a066-75dea0278b7f-kube-api-access-sxrmq\") pod \"maas-api-e2e-api-b-a22115-868758d68c-bgxsd\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.340542 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.340511    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.411755 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.411719    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:20.728654 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.728621    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"]
Sep 10 21:20:20.729252 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:20:20.729220    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e299c3_4337_4b31_a066_75dea0278b7f.slice/crio-e7b47d5d3e1a1420f15f6b7767b7d4f6ffd268ebb76fa3e6fbbc6e08b179011f WatchSource:0}: Error finding container e7b47d5d3e1a1420f15f6b7767b7d4f6ffd268ebb76fa3e6fbbc6e08b179011f: Status 404 returned error can't find the container with id e7b47d5d3e1a1420f15f6b7767b7d4f6ffd268ebb76fa3e6fbbc6e08b179011f
Sep 10 21:20:20.731204 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.731172    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:20:20.821147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.821109    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.946990 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:20.946952    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:21.742158 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:21.742117    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd" event={"ID":"93e299c3-4337-4b31-a066-75dea0278b7f","Type":"ContainerStarted","Data":"041d3234dcd3c9355ffb22e109f823ec67a447f7227bba1735b0d8b4215016b7"}
Sep 10 21:20:21.742640 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:21.742191    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd" event={"ID":"93e299c3-4337-4b31-a066-75dea0278b7f","Type":"ContainerStarted","Data":"e7b47d5d3e1a1420f15f6b7767b7d4f6ffd268ebb76fa3e6fbbc6e08b179011f"}
Sep 10 21:20:21.742640 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:21.742317    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:21.768593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:21.768538    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd" podStartSLOduration=1.357339197 podStartE2EDuration="1.768523274s" totalImagesPullingTime="411.184077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:21.767789617 +0000 UTC m=+1316.093550538" watchObservedRunningTime="2026-09-10 21:20:21.768523274 +0000 UTC m=+1316.094284195"
Sep 10 21:20:27.753398 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:27.753364    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:20:34.183385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:34.183345    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:38.959253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:38.959214    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:39.965049 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:39.965012    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:41.452350 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:41.452313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:43.757997 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:43.757963    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:46.710502 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:46.710465    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:51.171334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:51.171230    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:57.777568 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:20:57.777525    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:02.174454 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:02.174404    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:03.847889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:03.847853    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"]
Sep 10 21:21:03.849135 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:03.849092    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv" podUID="5d622e70-afd8-4b3c-a2d2-353d3a678726" containerName="maas-api" containerID="cri-o://06e2209d93641ba154b8e1ee015454bde7e316a8f4fc3318abecb8a777b5ad1c" gracePeriod=30
Sep 10 21:21:03.946674 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:03.946637    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:03.971764 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:03.971518    2584 generic.go:356] "Generic (PLEG): container finished" podID="5d622e70-afd8-4b3c-a2d2-353d3a678726" containerID="06e2209d93641ba154b8e1ee015454bde7e316a8f4fc3318abecb8a777b5ad1c" exitCode=0
Sep 10 21:21:03.971764 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:03.971688    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv" event={"ID":"5d622e70-afd8-4b3c-a2d2-353d3a678726","Type":"ContainerDied","Data":"06e2209d93641ba154b8e1ee015454bde7e316a8f4fc3318abecb8a777b5ad1c"}
Sep 10 21:21:03.985295 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:03.984637    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"]
Sep 10 21:21:03.985295 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:03.984900    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4" podUID="50213540-3b93-46bf-b2cc-04baacc5e175" containerName="payload-processing" containerID="cri-o://ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8" gracePeriod=30
Sep 10 21:21:04.109877 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.109850    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:21:04.117905 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.117880    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:04.169980 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.169951    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s27db\" (UniqueName: \"kubernetes.io/projected/5d622e70-afd8-4b3c-a2d2-353d3a678726-kube-api-access-s27db\") pod \"5d622e70-afd8-4b3c-a2d2-353d3a678726\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") "
Sep 10 21:21:04.170144 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.169988    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-metrics-certs\") pod \"5d622e70-afd8-4b3c-a2d2-353d3a678726\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") "
Sep 10 21:21:04.170144 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.170042    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls\") pod \"5d622e70-afd8-4b3c-a2d2-353d3a678726\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") "
Sep 10 21:21:04.170144 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.170059    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d622e70-afd8-4b3c-a2d2-353d3a678726-openshift-service-ca\") pod \"5d622e70-afd8-4b3c-a2d2-353d3a678726\" (UID: \"5d622e70-afd8-4b3c-a2d2-353d3a678726\") "
Sep 10 21:21:04.174456 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.170552    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d622e70-afd8-4b3c-a2d2-353d3a678726-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d622e70-afd8-4b3c-a2d2-353d3a678726" (UID: "5d622e70-afd8-4b3c-a2d2-353d3a678726"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:04.183526 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.182273    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d622e70-afd8-4b3c-a2d2-353d3a678726" (UID: "5d622e70-afd8-4b3c-a2d2-353d3a678726"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:21:04.183526 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.183332    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d622e70-afd8-4b3c-a2d2-353d3a678726-kube-api-access-s27db" (OuterVolumeSpecName: "kube-api-access-s27db") pod "5d622e70-afd8-4b3c-a2d2-353d3a678726" (UID: "5d622e70-afd8-4b3c-a2d2-353d3a678726"). InnerVolumeSpecName "kube-api-access-s27db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:04.184998 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.184943    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5d622e70-afd8-4b3c-a2d2-353d3a678726" (UID: "5d622e70-afd8-4b3c-a2d2-353d3a678726"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:21:04.244622 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.244598    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:21:04.276520 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.276433    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s27db\" (UniqueName: \"kubernetes.io/projected/5d622e70-afd8-4b3c-a2d2-353d3a678726-kube-api-access-s27db\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.276520 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.276467    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.276520 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.276481    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d622e70-afd8-4b3c-a2d2-353d3a678726-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.276520 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.276496    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d622e70-afd8-4b3c-a2d2-353d3a678726-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.338833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.338799    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:04.377872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.377790    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50213540-3b93-46bf-b2cc-04baacc5e175-plugins-config-volume\") pod \"50213540-3b93-46bf-b2cc-04baacc5e175\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") "
Sep 10 21:21:04.378023 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.377905    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stmdb\" (UniqueName: \"kubernetes.io/projected/50213540-3b93-46bf-b2cc-04baacc5e175-kube-api-access-stmdb\") pod \"50213540-3b93-46bf-b2cc-04baacc5e175\" (UID: \"50213540-3b93-46bf-b2cc-04baacc5e175\") "
Sep 10 21:21:04.378198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.378162    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50213540-3b93-46bf-b2cc-04baacc5e175-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50213540-3b93-46bf-b2cc-04baacc5e175" (UID: "50213540-3b93-46bf-b2cc-04baacc5e175"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:04.378395 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.378376    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50213540-3b93-46bf-b2cc-04baacc5e175-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.380084 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.380061    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50213540-3b93-46bf-b2cc-04baacc5e175-kube-api-access-stmdb" (OuterVolumeSpecName: "kube-api-access-stmdb") pod "50213540-3b93-46bf-b2cc-04baacc5e175" (UID: "50213540-3b93-46bf-b2cc-04baacc5e175"). InnerVolumeSpecName "kube-api-access-stmdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:04.479453 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.479400    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stmdb\" (UniqueName: \"kubernetes.io/projected/50213540-3b93-46bf-b2cc-04baacc5e175-kube-api-access-stmdb\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.504478 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.504443    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:04.834513 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.834479    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:04.979515 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.979474    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv" event={"ID":"5d622e70-afd8-4b3c-a2d2-353d3a678726","Type":"ContainerDied","Data":"1b6173b1e2375adca7eaf4668861b84c20084234153bcfedf99386f0d50f0603"}
Sep 10 21:21:04.979983 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.979480    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"
Sep 10 21:21:04.979983 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.979553    2584 scope.go:117] "RemoveContainer" containerID="06e2209d93641ba154b8e1ee015454bde7e316a8f4fc3318abecb8a777b5ad1c"
Sep 10 21:21:04.981091 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.981049    2584 generic.go:356] "Generic (PLEG): container finished" podID="50213540-3b93-46bf-b2cc-04baacc5e175" containerID="ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8" exitCode=0
Sep 10 21:21:04.981158 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.981093    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"
Sep 10 21:21:04.981158 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.981141    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4" event={"ID":"50213540-3b93-46bf-b2cc-04baacc5e175","Type":"ContainerDied","Data":"ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8"}
Sep 10 21:21:04.981299 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.981193    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4" event={"ID":"50213540-3b93-46bf-b2cc-04baacc5e175","Type":"ContainerDied","Data":"f47e51a894ee8ef279415c5e178541f496c085e3d290c78df1a5d3a70977652d"}
Sep 10 21:21:04.989074 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.989054    2584 scope.go:117] "RemoveContainer" containerID="ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8"
Sep 10 21:21:04.997149 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.997133    2584 scope.go:117] "RemoveContainer" containerID="ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8"
Sep 10 21:21:04.997432 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:21:04.997400    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8\": container with ID starting with ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8 not found: ID does not exist" containerID="ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8"
Sep 10 21:21:04.997513 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:04.997452    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8"} err="failed to get container status \"ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8\": rpc error: code = NotFound desc = could not find container \"ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8\": container with ID starting with ab9f662ff2dbe8641553579468644a3164b6489b78290faa4e38589c858ba6a8 not found: ID does not exist"
Sep 10 21:21:05.016811 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:05.016788    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"]
Sep 10 21:21:05.021952 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:05.021931    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5a30bd-569965ddc5-qz6rv"]
Sep 10 21:21:05.043611 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:05.043582    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"]
Sep 10 21:21:05.050005 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:05.049981    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5a30bd-54b69c8f5c-xqhp4"]
Sep 10 21:21:06.263209 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:06.263171    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:06.301731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:06.301701    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50213540-3b93-46bf-b2cc-04baacc5e175" path="/var/lib/kubelet/pods/50213540-3b93-46bf-b2cc-04baacc5e175/volumes"
Sep 10 21:21:06.302184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:06.302162    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d622e70-afd8-4b3c-a2d2-353d3a678726" path="/var/lib/kubelet/pods/5d622e70-afd8-4b3c-a2d2-353d3a678726/volumes"
Sep 10 21:21:11.447999 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:11.447933    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:11.598645 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:11.598611    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:11.741203 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:11.741111    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:12.248389 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:12.248358    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:18.356834 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.356800    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"]
Sep 10 21:21:18.357449 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.357412    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50213540-3b93-46bf-b2cc-04baacc5e175" containerName="payload-processing"
Sep 10 21:21:18.357549 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.357451    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="50213540-3b93-46bf-b2cc-04baacc5e175" containerName="payload-processing"
Sep 10 21:21:18.357549 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.357465    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d622e70-afd8-4b3c-a2d2-353d3a678726" containerName="maas-api"
Sep 10 21:21:18.357549 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.357472    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d622e70-afd8-4b3c-a2d2-353d3a678726" containerName="maas-api"
Sep 10 21:21:18.357694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.357603    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="50213540-3b93-46bf-b2cc-04baacc5e175" containerName="payload-processing"
Sep 10 21:21:18.357694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.357617    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d622e70-afd8-4b3c-a2d2-353d3a678726" containerName="maas-api"
Sep 10 21:21:18.362152 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.362131    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.366669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.366638    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"kube-root-ca.crt\""
Sep 10 21:21:18.366669 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.366653    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"openshift-service-ca.crt\""
Sep 10 21:21:18.366872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.366643    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"rate-test-model-755926-kserve-self-signed-certs\""
Sep 10 21:21:18.366872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.366644    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"default-dockercfg-t6cvs\""
Sep 10 21:21:18.375529 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.375500    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"]
Sep 10 21:21:18.394495 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.394453    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:18.519155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.519119    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tls-certs\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.519155 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.519155    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms82x\" (UniqueName: \"kubernetes.io/projected/d7b2c04a-c9b8-40de-82f4-041fce45f87e-kube-api-access-ms82x\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.519366 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.519272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-model-cache\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.519366 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.519301    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-home\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.519462 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.519381    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-dshm\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.519462 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.519443    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tmp-dir\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620270    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tls-certs\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms82x\" (UniqueName: \"kubernetes.io/projected/d7b2c04a-c9b8-40de-82f4-041fce45f87e-kube-api-access-ms82x\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620366    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-model-cache\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620394    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-home\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-dshm\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620520    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tmp-dir\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620949 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620917    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-home\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620949 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620931    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-model-cache\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.620949 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.620944    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tmp-dir\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.622646 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.622616    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-dshm\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.622930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.622908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tls-certs\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.630576 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.630553    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms82x\" (UniqueName: \"kubernetes.io/projected/d7b2c04a-c9b8-40de-82f4-041fce45f87e-kube-api-access-ms82x\") pod \"rate-test-model-755926-kserve-6cf68b6755-vsjpq\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") " pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.671624 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.671595    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:18.953662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:18.953629    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"]
Sep 10 21:21:18.955095 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:21:18.955067    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7b2c04a_c9b8_40de_82f4_041fce45f87e.slice/crio-6ee0935ae0e6b267f4a638a898436b5286988d60460d0ebe4e842fdd67b19d93 WatchSource:0}: Error finding container 6ee0935ae0e6b267f4a638a898436b5286988d60460d0ebe4e842fdd67b19d93: Status 404 returned error can't find the container with id 6ee0935ae0e6b267f4a638a898436b5286988d60460d0ebe4e842fdd67b19d93
Sep 10 21:21:19.066104 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:19.066062    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq" event={"ID":"d7b2c04a-c9b8-40de-82f4-041fce45f87e","Type":"ContainerStarted","Data":"b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6"}
Sep 10 21:21:19.066104 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:19.066109    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq" event={"ID":"d7b2c04a-c9b8-40de-82f4-041fce45f87e","Type":"ContainerStarted","Data":"6ee0935ae0e6b267f4a638a898436b5286988d60460d0ebe4e842fdd67b19d93"}
Sep 10 21:21:24.355867 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:24.355829    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:26.361567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:26.361529    2584 scope.go:117] "RemoveContainer" containerID="9a5a03d7c05f1dfbdec4468fb8700157ca534d9ec600d71fb57d68762fd044cf"
Sep 10 21:21:27.852927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:27.852872    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq" podStartSLOduration=9.852854135 podStartE2EDuration="9.852854135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:19.097691483 +0000 UTC m=+1373.423452404" watchObservedRunningTime="2026-09-10 21:21:27.852854135 +0000 UTC m=+1382.178615056"
Sep 10 21:21:27.853302 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:27.853227    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"]
Sep 10 21:21:27.853446 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:27.853413    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd" podUID="93e299c3-4337-4b31-a066-75dea0278b7f" containerName="maas-api" containerID="cri-o://041d3234dcd3c9355ffb22e109f823ec67a447f7227bba1735b0d8b4215016b7" gracePeriod=30
Sep 10 21:21:27.907277 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:27.907244    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:28.061550 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.061520    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:28.118071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.117949    2584 generic.go:356] "Generic (PLEG): container finished" podID="93e299c3-4337-4b31-a066-75dea0278b7f" containerID="041d3234dcd3c9355ffb22e109f823ec67a447f7227bba1735b0d8b4215016b7" exitCode=0
Sep 10 21:21:28.118071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.118026    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd" event={"ID":"93e299c3-4337-4b31-a066-75dea0278b7f","Type":"ContainerDied","Data":"041d3234dcd3c9355ffb22e109f823ec67a447f7227bba1735b0d8b4215016b7"}
Sep 10 21:21:28.118264 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.118072    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd" event={"ID":"93e299c3-4337-4b31-a066-75dea0278b7f","Type":"ContainerDied","Data":"e7b47d5d3e1a1420f15f6b7767b7d4f6ffd268ebb76fa3e6fbbc6e08b179011f"}
Sep 10 21:21:28.118264 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.118090    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7b47d5d3e1a1420f15f6b7767b7d4f6ffd268ebb76fa3e6fbbc6e08b179011f"
Sep 10 21:21:28.118264 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.118143    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:21:28.210962 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.210930    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-maas-api-tls\") pod \"93e299c3-4337-4b31-a066-75dea0278b7f\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") "
Sep 10 21:21:28.211130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.210985    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-metrics-certs\") pod \"93e299c3-4337-4b31-a066-75dea0278b7f\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") "
Sep 10 21:21:28.211130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.211092    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxrmq\" (UniqueName: \"kubernetes.io/projected/93e299c3-4337-4b31-a066-75dea0278b7f-kube-api-access-sxrmq\") pod \"93e299c3-4337-4b31-a066-75dea0278b7f\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") "
Sep 10 21:21:28.211204 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.211150    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e299c3-4337-4b31-a066-75dea0278b7f-openshift-service-ca\") pod \"93e299c3-4337-4b31-a066-75dea0278b7f\" (UID: \"93e299c3-4337-4b31-a066-75dea0278b7f\") "
Sep 10 21:21:28.211558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.211522    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e299c3-4337-4b31-a066-75dea0278b7f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93e299c3-4337-4b31-a066-75dea0278b7f" (UID: "93e299c3-4337-4b31-a066-75dea0278b7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:28.213147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.213128    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "93e299c3-4337-4b31-a066-75dea0278b7f" (UID: "93e299c3-4337-4b31-a066-75dea0278b7f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:21:28.213542 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.213526    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93e299c3-4337-4b31-a066-75dea0278b7f" (UID: "93e299c3-4337-4b31-a066-75dea0278b7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:21:28.213731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.213713    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e299c3-4337-4b31-a066-75dea0278b7f-kube-api-access-sxrmq" (OuterVolumeSpecName: "kube-api-access-sxrmq") pod "93e299c3-4337-4b31-a066-75dea0278b7f" (UID: "93e299c3-4337-4b31-a066-75dea0278b7f"). InnerVolumeSpecName "kube-api-access-sxrmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:28.259305 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.259269    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:28.312206 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.312173    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxrmq\" (UniqueName: \"kubernetes.io/projected/93e299c3-4337-4b31-a066-75dea0278b7f-kube-api-access-sxrmq\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:28.312323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.312211    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93e299c3-4337-4b31-a066-75dea0278b7f-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:28.312323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.312228    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:28.312323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.312242    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93e299c3-4337-4b31-a066-75dea0278b7f-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:28.591326 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.591291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:28.672391 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.672347    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:28.672598 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.672425    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:28.685581 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.685551    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:28.903907 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:28.903863    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:29.122594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:29.122562    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"
Sep 10 21:21:29.134319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:29.134295    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:21:29.159861 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:29.159791    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"]
Sep 10 21:21:29.180065 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:29.180037    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a22115-868758d68c-bgxsd"]
Sep 10 21:21:30.301774 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:30.301731    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93e299c3-4337-4b31-a066-75dea0278b7f" path="/var/lib/kubelet/pods/93e299c3-4337-4b31-a066-75dea0278b7f/volumes"
Sep 10 21:21:35.783117 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:35.783087    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:35.867828 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:35.867791    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:36.069247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:36.069156    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:36.711700 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:36.711664    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:47.101716 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.101681    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"]
Sep 10 21:21:47.102193 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.101889    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg" podUID="91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" containerName="maas-api" containerID="cri-o://da63e356965f140e0c9ab452a295b7cb184daff377308b96b11d8fb685119df5" gracePeriod=30
Sep 10 21:21:47.156244 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.156205    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:47.218142 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.218114    2584 generic.go:356] "Generic (PLEG): container finished" podID="91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" containerID="da63e356965f140e0c9ab452a295b7cb184daff377308b96b11d8fb685119df5" exitCode=0
Sep 10 21:21:47.218267 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.218185    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg" event={"ID":"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2","Type":"ContainerDied","Data":"da63e356965f140e0c9ab452a295b7cb184daff377308b96b11d8fb685119df5"}
Sep 10 21:21:47.283767 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.283733    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:47.362681 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.362620    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:21:47.383570 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.383509    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:47.482310 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.482274    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:47.502762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.502724    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-metrics-certs\") pod \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") "
Sep 10 21:21:47.502762 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.502763    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls\") pod \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") "
Sep 10 21:21:47.503002 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.502840    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwh9n\" (UniqueName: \"kubernetes.io/projected/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-kube-api-access-xwh9n\") pod \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") "
Sep 10 21:21:47.503002 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.502868    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-openshift-service-ca\") pod \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\" (UID: \"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2\") "
Sep 10 21:21:47.503357 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.503324    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" (UID: "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:47.505313 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.505264    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" (UID: "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:21:47.505460 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.505370    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" (UID: "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:21:47.505704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.505685    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-kube-api-access-xwh9n" (OuterVolumeSpecName: "kube-api-access-xwh9n") pod "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" (UID: "91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2"). InnerVolumeSpecName "kube-api-access-xwh9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:47.603922 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.603886    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:47.603922 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.603914    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:47.603922 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.603923    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwh9n\" (UniqueName: \"kubernetes.io/projected/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-kube-api-access-xwh9n\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:47.603922 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.603932    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:21:47.802035 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:47.801997    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:48.224815 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:48.224785    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"
Sep 10 21:21:48.225293 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:48.224782    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg" event={"ID":"91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2","Type":"ContainerDied","Data":"3a186ec00478eee06b2b61872089873fc6bef4fdda9cbe261ab95f041b89121e"}
Sep 10 21:21:48.225293 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:48.224943    2584 scope.go:117] "RemoveContainer" containerID="da63e356965f140e0c9ab452a295b7cb184daff377308b96b11d8fb685119df5"
Sep 10 21:21:48.249744 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:48.249710    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"]
Sep 10 21:21:48.251889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:48.251864    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-290e03-77d7c6697d-7lprg"]
Sep 10 21:21:48.305170 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:48.305136    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" path="/var/lib/kubelet/pods/91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2/volumes"
Sep 10 21:21:54.362207 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:54.362173    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:54.580120 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:54.580086    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:54.806700 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:54.806660    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:55.200030 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:21:55.200004    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:03.517296 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:03.517216    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:04.433183 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:04.433147    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"]
Sep 10 21:22:04.460708 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:04.460670    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:04.861265 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:04.861168    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:05.177863 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.177824    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:05.344907 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.344877    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:05.635489 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.635402    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:05.662435 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.662390    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"]
Sep 10 21:22:05.662686 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.662661    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk" podUID="9b0b36db-32ec-4618-8652-d44a1761655d" containerName="payload-processing" containerID="cri-o://2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552" gracePeriod=30
Sep 10 21:22:05.671494 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.670906    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"]
Sep 10 21:22:05.671494 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.671118    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x" podUID="2a9e2e13-3716-4ce9-b682-cc77fc9b602e" containerName="payload-pre-processing" containerID="cri-o://077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a" gracePeriod=30
Sep 10 21:22:05.707753 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.707719    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:05.810590 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.810555    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:05.892665 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.892618    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:05.928460 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.928437    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:22:05.959809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.959782    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:22:05.985272 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.985243    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7mnm\" (UniqueName: \"kubernetes.io/projected/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-kube-api-access-k7mnm\") pod \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") "
Sep 10 21:22:05.985468 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.985316    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p88p6\" (UniqueName: \"kubernetes.io/projected/9b0b36db-32ec-4618-8652-d44a1761655d-kube-api-access-p88p6\") pod \"9b0b36db-32ec-4618-8652-d44a1761655d\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") "
Sep 10 21:22:05.985468 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.985373    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-plugins-config-volume\") pod \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\" (UID: \"2a9e2e13-3716-4ce9-b682-cc77fc9b602e\") "
Sep 10 21:22:05.985606 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.985468    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b0b36db-32ec-4618-8652-d44a1761655d-plugins-config-volume\") pod \"9b0b36db-32ec-4618-8652-d44a1761655d\" (UID: \"9b0b36db-32ec-4618-8652-d44a1761655d\") "
Sep 10 21:22:05.985821 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.985788    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a9e2e13-3716-4ce9-b682-cc77fc9b602e" (UID: "2a9e2e13-3716-4ce9-b682-cc77fc9b602e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:22:05.985952 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.985880    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b0b36db-32ec-4618-8652-d44a1761655d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b0b36db-32ec-4618-8652-d44a1761655d" (UID: "9b0b36db-32ec-4618-8652-d44a1761655d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:22:05.987501 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.987472    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-kube-api-access-k7mnm" (OuterVolumeSpecName: "kube-api-access-k7mnm") pod "2a9e2e13-3716-4ce9-b682-cc77fc9b602e" (UID: "2a9e2e13-3716-4ce9-b682-cc77fc9b602e"). InnerVolumeSpecName "kube-api-access-k7mnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:22:05.987908 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:05.987877    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b0b36db-32ec-4618-8652-d44a1761655d-kube-api-access-p88p6" (OuterVolumeSpecName: "kube-api-access-p88p6") pod "9b0b36db-32ec-4618-8652-d44a1761655d" (UID: "9b0b36db-32ec-4618-8652-d44a1761655d"). InnerVolumeSpecName "kube-api-access-p88p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:22:06.086582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.086546    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b0b36db-32ec-4618-8652-d44a1761655d-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.086582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.086576    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7mnm\" (UniqueName: \"kubernetes.io/projected/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-kube-api-access-k7mnm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.086582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.086585    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p88p6\" (UniqueName: \"kubernetes.io/projected/9b0b36db-32ec-4618-8652-d44a1761655d-kube-api-access-p88p6\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.086582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.086593    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a9e2e13-3716-4ce9-b682-cc77fc9b602e-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.215545 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.215512    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:06.322490 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.322462    2584 generic.go:356] "Generic (PLEG): container finished" podID="2a9e2e13-3716-4ce9-b682-cc77fc9b602e" containerID="077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a" exitCode=0
Sep 10 21:22:06.322597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.322503    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"
Sep 10 21:22:06.322597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.322545    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x" event={"ID":"2a9e2e13-3716-4ce9-b682-cc77fc9b602e","Type":"ContainerDied","Data":"077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a"}
Sep 10 21:22:06.322709 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.322599    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x" event={"ID":"2a9e2e13-3716-4ce9-b682-cc77fc9b602e","Type":"ContainerDied","Data":"48c7f5e4871d955830ef0a03bcca9e6103c99da1ac39cc4cb8c59e6ffb078274"}
Sep 10 21:22:06.322709 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.322630    2584 scope.go:117] "RemoveContainer" containerID="077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a"
Sep 10 21:22:06.323966 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.323943    2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0b36db-32ec-4618-8652-d44a1761655d" containerID="2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552" exitCode=0
Sep 10 21:22:06.324071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.323978    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"
Sep 10 21:22:06.324071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.324006    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk" event={"ID":"9b0b36db-32ec-4618-8652-d44a1761655d","Type":"ContainerDied","Data":"2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552"}
Sep 10 21:22:06.324071 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.324042    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk" event={"ID":"9b0b36db-32ec-4618-8652-d44a1761655d","Type":"ContainerDied","Data":"b850b93bdd4ef5a649c72ceb0e7f335f4b745ff78bad1faf6f0af57325163fa8"}
Sep 10 21:22:06.331251 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.331234    2584 scope.go:117] "RemoveContainer" containerID="077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a"
Sep 10 21:22:06.331523 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:22:06.331504    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a\": container with ID starting with 077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a not found: ID does not exist" containerID="077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a"
Sep 10 21:22:06.331618 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.331528    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a"} err="failed to get container status \"077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a\": rpc error: code = NotFound desc = could not find container \"077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a\": container with ID starting with 077e9b05ee1055040791fe214ad8f83369fabc72020a96591e1b22ddbbf2797a not found: ID does not exist"
Sep 10 21:22:06.331618 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.331542    2584 scope.go:117] "RemoveContainer" containerID="2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552"
Sep 10 21:22:06.339400 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.339384    2584 scope.go:117] "RemoveContainer" containerID="2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552"
Sep 10 21:22:06.339784 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:22:06.339759    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552\": container with ID starting with 2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552 not found: ID does not exist" containerID="2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552"
Sep 10 21:22:06.339875 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.339788    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552"} err="failed to get container status \"2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552\": rpc error: code = NotFound desc = could not find container \"2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552\": container with ID starting with 2b757ae1c553c3a2aa2ee074fa9b9de31e16ed50ff15fb2439eb8c2658a58552 not found: ID does not exist"
Sep 10 21:22:06.342564 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.342544    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"]
Sep 10 21:22:06.347932 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.347915    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-9f71ac-5bcf998866-db66x"]
Sep 10 21:22:06.359677 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.359654    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"]
Sep 10 21:22:06.371562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:06.371526    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-9f71ac-6696c5556f-pbnnk"]
Sep 10 21:22:08.301955 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:08.301917    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9e2e13-3716-4ce9-b682-cc77fc9b602e" path="/var/lib/kubelet/pods/2a9e2e13-3716-4ce9-b682-cc77fc9b602e/volumes"
Sep 10 21:22:08.302594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:08.302573    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b0b36db-32ec-4618-8652-d44a1761655d" path="/var/lib/kubelet/pods/9b0b36db-32ec-4618-8652-d44a1761655d/volumes"
Sep 10 21:22:08.334719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:08.334682    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:08.476346 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:08.476300    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:09.555623 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.555589    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"]
Sep 10 21:22:09.556205 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556112    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" containerName="maas-api"
Sep 10 21:22:09.556278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556231    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" containerName="maas-api"
Sep 10 21:22:09.556278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556257    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9e2e13-3716-4ce9-b682-cc77fc9b602e" containerName="payload-pre-processing"
Sep 10 21:22:09.556278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556267    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9e2e13-3716-4ce9-b682-cc77fc9b602e" containerName="payload-pre-processing"
Sep 10 21:22:09.556278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556276    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93e299c3-4337-4b31-a066-75dea0278b7f" containerName="maas-api"
Sep 10 21:22:09.556589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556286    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="93e299c3-4337-4b31-a066-75dea0278b7f" containerName="maas-api"
Sep 10 21:22:09.556589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556545    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0b36db-32ec-4618-8652-d44a1761655d" containerName="payload-processing"
Sep 10 21:22:09.556589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556590    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0b36db-32ec-4618-8652-d44a1761655d" containerName="payload-processing"
Sep 10 21:22:09.556746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556722    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="93e299c3-4337-4b31-a066-75dea0278b7f" containerName="maas-api"
Sep 10 21:22:09.556746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556738    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f20cbc-8f8c-4ba7-a52b-724d80a4f6d2" containerName="maas-api"
Sep 10 21:22:09.556819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556750    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0b36db-32ec-4618-8652-d44a1761655d" containerName="payload-processing"
Sep 10 21:22:09.556819 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.556761    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9e2e13-3716-4ce9-b682-cc77fc9b602e" containerName="payload-pre-processing"
Sep 10 21:22:09.561603 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.561582    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.564238 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.564219    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-84b69f-serving-cert\""
Sep 10 21:22:09.572965 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.572911    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"]
Sep 10 21:22:09.605660 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.605621    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:09.617777 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.617752    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-maas-api-tls\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.617901 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.617796    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdmk7\" (UniqueName: \"kubernetes.io/projected/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-kube-api-access-cdmk7\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.617901 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.617818    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.617901 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.617863    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-metrics-certs\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.709330 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.709272    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:09.719278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.719255    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-metrics-certs\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.719411 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.719350    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-maas-api-tls\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.719411 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.719395    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdmk7\" (UniqueName: \"kubernetes.io/projected/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-kube-api-access-cdmk7\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.719567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.719443    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.720040 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.720022    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.721768 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.721744    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-maas-api-tls\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.721871 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.721801    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-metrics-certs\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.728932 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.728908    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdmk7\" (UniqueName: \"kubernetes.io/projected/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-kube-api-access-cdmk7\") pod \"maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:09.803962 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.803924    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:09.872584 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:09.872488    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:10.126052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:10.125962    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:10.155985 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:10.155959    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"]
Sep 10 21:22:10.159744 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:22:10.159634    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d93f2db_90df_4cdc_9b4e_3bbe4694fdaa.slice/crio-b38c40aa42ea25600400008ecc3e4abc47ea461c140030025b7b42681db4a195 WatchSource:0}: Error finding container b38c40aa42ea25600400008ecc3e4abc47ea461c140030025b7b42681db4a195: Status 404 returned error can't find the container with id b38c40aa42ea25600400008ecc3e4abc47ea461c140030025b7b42681db4a195
Sep 10 21:22:10.228017 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:10.227970    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:10.349301 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:10.349266    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn" event={"ID":"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa","Type":"ContainerStarted","Data":"b38c40aa42ea25600400008ecc3e4abc47ea461c140030025b7b42681db4a195"}
Sep 10 21:22:11.360234 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:11.360196    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn" event={"ID":"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa","Type":"ContainerStarted","Data":"b0879fea7179d447299ee131b0256e4e4a1be7a90b2e4d35bbce748aab05ad1c"}
Sep 10 21:22:11.360746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:11.360468    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:11.380946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:11.380893    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn" podStartSLOduration=1.899883987 podStartE2EDuration="2.38087722s" totalImagesPullingTime="480.993233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:11.379157329 +0000 UTC m=+1425.704918253" watchObservedRunningTime="2026-09-10 21:22:11.38087722 +0000 UTC m=+1425.706638138"
Sep 10 21:22:12.182588 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:12.182554    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:12.282716 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:12.282681    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:12.448500 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:12.448396    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:17.371894 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:17.371860    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:22:19.451565 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.451473    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq" podUID="d7b2c04a-c9b8-40de-82f4-041fce45f87e" containerName="main" containerID="cri-o://b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6" gracePeriod=45
Sep 10 21:22:19.706885 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.706830    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:22:19.820463 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820402    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tmp-dir\") pod \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") "
Sep 10 21:22:19.820626 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820533    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-model-cache\") pod \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") "
Sep 10 21:22:19.820626 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820556    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-dshm\") pod \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") "
Sep 10 21:22:19.820626 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820596    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ms82x\" (UniqueName: \"kubernetes.io/projected/d7b2c04a-c9b8-40de-82f4-041fce45f87e-kube-api-access-ms82x\") pod \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") "
Sep 10 21:22:19.820746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820631    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tls-certs\") pod \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") "
Sep 10 21:22:19.820746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820660    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-home\") pod \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\" (UID: \"d7b2c04a-c9b8-40de-82f4-041fce45f87e\") "
Sep 10 21:22:19.820746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820673    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7b2c04a-c9b8-40de-82f4-041fce45f87e" (UID: "d7b2c04a-c9b8-40de-82f4-041fce45f87e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:19.820906 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820830    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7b2c04a-c9b8-40de-82f4-041fce45f87e" (UID: "d7b2c04a-c9b8-40de-82f4-041fce45f87e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:19.821004 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.820981    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-home" (OuterVolumeSpecName: "home") pod "d7b2c04a-c9b8-40de-82f4-041fce45f87e" (UID: "d7b2c04a-c9b8-40de-82f4-041fce45f87e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:19.821058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.821045    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-model-cache\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:19.821100 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.821064    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tmp-dir\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:19.822887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.822866    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-dshm" (OuterVolumeSpecName: "dshm") pod "d7b2c04a-c9b8-40de-82f4-041fce45f87e" (UID: "d7b2c04a-c9b8-40de-82f4-041fce45f87e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:19.822887 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.822878    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7b2c04a-c9b8-40de-82f4-041fce45f87e" (UID: "d7b2c04a-c9b8-40de-82f4-041fce45f87e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:22:19.823042 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.822898    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7b2c04a-c9b8-40de-82f4-041fce45f87e-kube-api-access-ms82x" (OuterVolumeSpecName: "kube-api-access-ms82x") pod "d7b2c04a-c9b8-40de-82f4-041fce45f87e" (UID: "d7b2c04a-c9b8-40de-82f4-041fce45f87e"). InnerVolumeSpecName "kube-api-access-ms82x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:22:19.921693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.921657    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms82x\" (UniqueName: \"kubernetes.io/projected/d7b2c04a-c9b8-40de-82f4-041fce45f87e-kube-api-access-ms82x\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:19.921693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.921686    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7b2c04a-c9b8-40de-82f4-041fce45f87e-tls-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:19.921693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.921696    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-home\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:19.921693 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:19.921703    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7b2c04a-c9b8-40de-82f4-041fce45f87e-dshm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:22:20.408136 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.408101    2584 generic.go:356] "Generic (PLEG): container finished" podID="d7b2c04a-c9b8-40de-82f4-041fce45f87e" containerID="b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6" exitCode=0
Sep 10 21:22:20.408323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.408145    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"
Sep 10 21:22:20.408323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.408189    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq" event={"ID":"d7b2c04a-c9b8-40de-82f4-041fce45f87e","Type":"ContainerDied","Data":"b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6"}
Sep 10 21:22:20.408323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.408232    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq" event={"ID":"d7b2c04a-c9b8-40de-82f4-041fce45f87e","Type":"ContainerDied","Data":"6ee0935ae0e6b267f4a638a898436b5286988d60460d0ebe4e842fdd67b19d93"}
Sep 10 21:22:20.408323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.408252    2584 scope.go:117] "RemoveContainer" containerID="b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6"
Sep 10 21:22:20.416765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.416747    2584 scope.go:117] "RemoveContainer" containerID="b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6"
Sep 10 21:22:20.417054 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:22:20.417034    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6\": container with ID starting with b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6 not found: ID does not exist" containerID="b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6"
Sep 10 21:22:20.417110 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.417062    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6"} err="failed to get container status \"b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6\": rpc error: code = NotFound desc = could not find container \"b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6\": container with ID starting with b7b4e2286ef6ec034b883be084580656b6e46af091f290eac0d3c420dc3920f6 not found: ID does not exist"
Sep 10 21:22:20.425665 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.425634    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"]
Sep 10 21:22:20.429301 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:20.429278    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/rate-test-model-755926-kserve-6cf68b6755-vsjpq"]
Sep 10 21:22:22.301855 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:22.301817    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7b2c04a-c9b8-40de-82f4-041fce45f87e" path="/var/lib/kubelet/pods/d7b2c04a-c9b8-40de-82f4-041fce45f87e/volumes"
Sep 10 21:22:33.235805 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:33.235771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:33.885738 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:33.885702    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:34.203834 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:34.203792    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:34.326050 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:34.326012    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:37.520459 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:37.520410    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:37.802234 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:37.802140    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:38.785881 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.785846    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"]
Sep 10 21:22:38.786318 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.786301    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7b2c04a-c9b8-40de-82f4-041fce45f87e" containerName="main"
Sep 10 21:22:38.786318 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.786316    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7b2c04a-c9b8-40de-82f4-041fce45f87e" containerName="main"
Sep 10 21:22:38.786393 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.786384    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7b2c04a-c9b8-40de-82f4-041fce45f87e" containerName="main"
Sep 10 21:22:38.790960 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.790939    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:38.793739 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.793707    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-868994-serving-cert\""
Sep 10 21:22:38.819804 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.819742    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"]
Sep 10 21:22:38.878736 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.878698    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:38.903845 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.903806    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgn9g\" (UniqueName: \"kubernetes.io/projected/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-kube-api-access-wgn9g\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:38.904050 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.903948    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-metrics-certs\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:38.904050 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.904006    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:38.904178 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:38.904070    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.005216 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.005181    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgn9g\" (UniqueName: \"kubernetes.io/projected/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-kube-api-access-wgn9g\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.005405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.005261    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-metrics-certs\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.005405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.005294    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.005405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.005317    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.006042 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.006018    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.007866 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.007839    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-metrics-certs\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.007959 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.007889    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.012185 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.012160    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:39.020542 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.020520    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgn9g\" (UniqueName: \"kubernetes.io/projected/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-kube-api-access-wgn9g\") pod \"maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.103875 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.103778    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:39.125953 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.125912    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:39.392873 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.392765    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"]
Sep 10 21:22:39.397233 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:22:39.397206    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbc7f38_2281_4a0b_a9d5_1ad3f999aa3c.slice/crio-1dc91f3356252f31ff8210e20c83547f806861b2f33eb65097cc77a2c12e9236 WatchSource:0}: Error finding container 1dc91f3356252f31ff8210e20c83547f806861b2f33eb65097cc77a2c12e9236: Status 404 returned error can't find the container with id 1dc91f3356252f31ff8210e20c83547f806861b2f33eb65097cc77a2c12e9236
Sep 10 21:22:39.507774 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.507739    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5" event={"ID":"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c","Type":"ContainerStarted","Data":"1dc91f3356252f31ff8210e20c83547f806861b2f33eb65097cc77a2c12e9236"}
Sep 10 21:22:39.696635 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.696594    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:39.849863 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.849835    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:39.885585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.885555    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"]
Sep 10 21:22:39.890441 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.890393    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:39.892605 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.892582    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-868994-dockercfg-gq8zd\""
Sep 10 21:22:39.894209 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.894190    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-868994\""
Sep 10 21:22:39.924037 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:39.924003    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"]
Sep 10 21:22:40.014157 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.014054    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rvf8\" (UniqueName: \"kubernetes.io/projected/d781d605-879a-4065-8d79-f754bd2e25f8-kube-api-access-4rvf8\") pod \"payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:40.014306 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.014207    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d781d605-879a-4065-8d79-f754bd2e25f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:40.115461 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.115407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d781d605-879a-4065-8d79-f754bd2e25f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:40.115648 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.115540    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rvf8\" (UniqueName: \"kubernetes.io/projected/d781d605-879a-4065-8d79-f754bd2e25f8-kube-api-access-4rvf8\") pod \"payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:40.116210 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.116186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d781d605-879a-4065-8d79-f754bd2e25f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:40.153799 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.153762    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rvf8\" (UniqueName: \"kubernetes.io/projected/d781d605-879a-4065-8d79-f754bd2e25f8-kube-api-access-4rvf8\") pod \"payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:40.202287 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.202243    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:40.489398 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.489359    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"]
Sep 10 21:22:40.493477 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:22:40.493447    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd781d605_879a_4065_8d79_f754bd2e25f8.slice/crio-39846080c0b742b19abc8b87c781026ed1f154f19f7882cd7b81742f4b56b53b WatchSource:0}: Error finding container 39846080c0b742b19abc8b87c781026ed1f154f19f7882cd7b81742f4b56b53b: Status 404 returned error can't find the container with id 39846080c0b742b19abc8b87c781026ed1f154f19f7882cd7b81742f4b56b53b
Sep 10 21:22:40.515779 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.515748    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5" event={"ID":"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c","Type":"ContainerStarted","Data":"e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83"}
Sep 10 21:22:40.515912 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.515859    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:40.517116 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.517095    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw" event={"ID":"d781d605-879a-4065-8d79-f754bd2e25f8","Type":"ContainerStarted","Data":"39846080c0b742b19abc8b87c781026ed1f154f19f7882cd7b81742f4b56b53b"}
Sep 10 21:22:40.545115 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:40.545064    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5" podStartSLOduration=2.0936974250000002 podStartE2EDuration="2.545048648s" totalImagesPullingTime="451.351223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:40.544112122 +0000 UTC m=+1454.869873044" watchObservedRunningTime="2026-09-10 21:22:40.545048648 +0000 UTC m=+1454.870809610"
Sep 10 21:22:41.524177 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:41.524131    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw" event={"ID":"d781d605-879a-4065-8d79-f754bd2e25f8","Type":"ContainerStarted","Data":"701b5e079863fa11a2e65f3b24f3a39bad224073289d7c4f59cf6aad62ab685e"}
Sep 10 21:22:41.524634 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:41.524395    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:41.547211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:41.547148    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw" podStartSLOduration=2.547131748 podStartE2EDuration="2.547131748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:41.545787809 +0000 UTC m=+1455.871548731" watchObservedRunningTime="2026-09-10 21:22:41.547131748 +0000 UTC m=+1455.872892711"
Sep 10 21:22:46.530236 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:46.530208    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:22:52.531592 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:52.531561    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:22:57.598931 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:22:57.598898    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:00.334784 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:00.334743    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:02.396692 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:02.396652    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:15.153324 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:15.153288    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.225710 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:26.225682    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:23:26.226376 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:26.226354    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:23:30.187835 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:30.187787    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:49.099340 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:49.099246    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:50.476360 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:50.476323    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:52.930636 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:52.930600    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:53.031966 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:53.031921    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:54.062211 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.062179    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"]
Sep 10 21:23:54.066176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.066151    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.073406 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.073380    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-6e4b80-serving-cert\""
Sep 10 21:23:54.081811 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.081785    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"]
Sep 10 21:23:54.111924 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.111876    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:54.212800 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.212763    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298515e-e196-4b00-bb34-29601d4b25d0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.212981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.212820    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqc5g\" (UniqueName: \"kubernetes.io/projected/1298515e-e196-4b00-bb34-29601d4b25d0-kube-api-access-dqc5g\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.212981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.212886    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.212981 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.212949    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.228191 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.228161    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:54.314273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.314191    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqc5g\" (UniqueName: \"kubernetes.io/projected/1298515e-e196-4b00-bb34-29601d4b25d0-kube-api-access-dqc5g\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.314273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.314230    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.314273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.314274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.314562 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:23:54.314393    2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-serving-cert: secret "maas-api-e2e-ipp-cleanup-6e4b80-serving-cert" not found
Sep 10 21:23:54.314562 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.314451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298515e-e196-4b00-bb34-29601d4b25d0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.314562 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:23:54.314489    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls podName:1298515e-e196-4b00-bb34-29601d4b25d0 nodeName:}" failed. No retries permitted until 2026-09-10 21:23:54.814468294 +0000 UTC m=+1529.140229209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w" (UID: "1298515e-e196-4b00-bb34-29601d4b25d0") : secret "maas-api-e2e-ipp-cleanup-6e4b80-serving-cert" not found
Sep 10 21:23:54.315025 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.314995    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298515e-e196-4b00-bb34-29601d4b25d0-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.316674 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.316647    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.325154 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.325133    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqc5g\" (UniqueName: \"kubernetes.io/projected/1298515e-e196-4b00-bb34-29601d4b25d0-kube-api-access-dqc5g\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.440786 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.440748    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:54.613809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.613716    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:54.712689 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.712650    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:54.819718 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.819674    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.822468 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.822411    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:54.980036 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:54.979991    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:55.255992 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:55.255960    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"]
Sep 10 21:23:55.257906 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:23:55.257874    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1298515e_e196_4b00_bb34_29601d4b25d0.slice/crio-87b5fe74a8db768b5719567f28fef533605b034dbc567f3cd85a2dc449b76f63 WatchSource:0}: Error finding container 87b5fe74a8db768b5719567f28fef533605b034dbc567f3cd85a2dc449b76f63: Status 404 returned error can't find the container with id 87b5fe74a8db768b5719567f28fef533605b034dbc567f3cd85a2dc449b76f63
Sep 10 21:23:55.915796 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:55.915760    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w" event={"ID":"1298515e-e196-4b00-bb34-29601d4b25d0","Type":"ContainerStarted","Data":"b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2"}
Sep 10 21:23:55.915986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:55.915812    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w" event={"ID":"1298515e-e196-4b00-bb34-29601d4b25d0","Type":"ContainerStarted","Data":"87b5fe74a8db768b5719567f28fef533605b034dbc567f3cd85a2dc449b76f63"}
Sep 10 21:23:55.915986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:55.915874    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:23:55.935193 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:23:55.935086    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w" podStartSLOduration=1.515084286 podStartE2EDuration="1.935069754s" totalImagesPullingTime="419.985468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:55.933861689 +0000 UTC m=+1530.259622614" watchObservedRunningTime="2026-09-10 21:23:55.935069754 +0000 UTC m=+1530.260830676"
Sep 10 21:24:01.926732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:01.926690    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:24:17.908213 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:17.908178    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:24.660627 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.660587    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"]
Sep 10 21:24:24.661305 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.660784    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w" podUID="1298515e-e196-4b00-bb34-29601d4b25d0" containerName="maas-api" containerID="cri-o://b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2" gracePeriod=30
Sep 10 21:24:24.686590 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.686558    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:24.767844 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.767806    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:24.838969 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.838929    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:24.907089 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.907053    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:24.932637 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.932613    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:24:24.984020 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:24.983984    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:25.019024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.018991    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-metrics-certs\") pod \"1298515e-e196-4b00-bb34-29601d4b25d0\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") "
Sep 10 21:24:25.019024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.019029    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298515e-e196-4b00-bb34-29601d4b25d0-openshift-service-ca\") pod \"1298515e-e196-4b00-bb34-29601d4b25d0\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") "
Sep 10 21:24:25.019273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.019073    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqc5g\" (UniqueName: \"kubernetes.io/projected/1298515e-e196-4b00-bb34-29601d4b25d0-kube-api-access-dqc5g\") pod \"1298515e-e196-4b00-bb34-29601d4b25d0\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") "
Sep 10 21:24:25.019273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.019148    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls\") pod \"1298515e-e196-4b00-bb34-29601d4b25d0\" (UID: \"1298515e-e196-4b00-bb34-29601d4b25d0\") "
Sep 10 21:24:25.019486 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.019453    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1298515e-e196-4b00-bb34-29601d4b25d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1298515e-e196-4b00-bb34-29601d4b25d0" (UID: "1298515e-e196-4b00-bb34-29601d4b25d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:25.021302 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.021276    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1298515e-e196-4b00-bb34-29601d4b25d0" (UID: "1298515e-e196-4b00-bb34-29601d4b25d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:25.021407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.021377    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1298515e-e196-4b00-bb34-29601d4b25d0-kube-api-access-dqc5g" (OuterVolumeSpecName: "kube-api-access-dqc5g") pod "1298515e-e196-4b00-bb34-29601d4b25d0" (UID: "1298515e-e196-4b00-bb34-29601d4b25d0"). InnerVolumeSpecName "kube-api-access-dqc5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:25.021407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.021396    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1298515e-e196-4b00-bb34-29601d4b25d0" (UID: "1298515e-e196-4b00-bb34-29601d4b25d0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:25.068812 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.068777    2584 generic.go:356] "Generic (PLEG): container finished" podID="1298515e-e196-4b00-bb34-29601d4b25d0" containerID="b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2" exitCode=0
Sep 10 21:24:25.068986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.068819    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w" event={"ID":"1298515e-e196-4b00-bb34-29601d4b25d0","Type":"ContainerDied","Data":"b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2"}
Sep 10 21:24:25.068986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.068858    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"
Sep 10 21:24:25.068986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.068886    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w" event={"ID":"1298515e-e196-4b00-bb34-29601d4b25d0","Type":"ContainerDied","Data":"87b5fe74a8db768b5719567f28fef533605b034dbc567f3cd85a2dc449b76f63"}
Sep 10 21:24:25.068986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.068916    2584 scope.go:117] "RemoveContainer" containerID="b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2"
Sep 10 21:24:25.079396 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.079372    2584 scope.go:117] "RemoveContainer" containerID="b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2"
Sep 10 21:24:25.079730 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:24:25.079699    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2\": container with ID starting with b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2 not found: ID does not exist" containerID="b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2"
Sep 10 21:24:25.079851 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.079739    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2"} err="failed to get container status \"b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2\": rpc error: code = NotFound desc = could not find container \"b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2\": container with ID starting with b6b5e19bacba03dfe0bd0192fbfe3cda6791836b5178aee789df3b631d1d80f2 not found: ID does not exist"
Sep 10 21:24:25.096341 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.096311    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"]
Sep 10 21:24:25.100089 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.100063    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-6e4b80-894db74f5-hsg9w"]
Sep 10 21:24:25.120375 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.120344    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqc5g\" (UniqueName: \"kubernetes.io/projected/1298515e-e196-4b00-bb34-29601d4b25d0-kube-api-access-dqc5g\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:25.120375 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.120370    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:25.120553 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.120387    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1298515e-e196-4b00-bb34-29601d4b25d0-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:25.120553 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.120399    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1298515e-e196-4b00-bb34-29601d4b25d0-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:25.301256 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:25.301174    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:26.302506 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:26.302474    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1298515e-e196-4b00-bb34-29601d4b25d0" path="/var/lib/kubelet/pods/1298515e-e196-4b00-bb34-29601d4b25d0/volumes"
Sep 10 21:24:31.551811 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:31.551771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:31.657672 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:31.657637    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:31.812244 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:31.812151    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:35.558910 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:35.558867    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:45.815447 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:45.815399    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:46.119396 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:46.119282    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:50.987852 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:50.987815    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"]
Sep 10 21:24:50.988335 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:50.988066    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn" podUID="1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" containerName="maas-api" containerID="cri-o://b0879fea7179d447299ee131b0256e4e4a1be7a90b2e4d35bbce748aab05ad1c" gracePeriod=30
Sep 10 21:24:51.015373 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.015335    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:51.106251 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.106144    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:51.187025 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.186965    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:51.213253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.213222    2584 generic.go:356] "Generic (PLEG): container finished" podID="1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" containerID="b0879fea7179d447299ee131b0256e4e4a1be7a90b2e4d35bbce748aab05ad1c" exitCode=0
Sep 10 21:24:51.213459 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.213299    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn" event={"ID":"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa","Type":"ContainerDied","Data":"b0879fea7179d447299ee131b0256e4e4a1be7a90b2e4d35bbce748aab05ad1c"}
Sep 10 21:24:51.245051 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.244997    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:24:51.275279 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.275244    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-openshift-service-ca\") pod \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") "
Sep 10 21:24:51.275475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.275302    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-metrics-certs\") pod \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") "
Sep 10 21:24:51.275475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.275402    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdmk7\" (UniqueName: \"kubernetes.io/projected/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-kube-api-access-cdmk7\") pod \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") "
Sep 10 21:24:51.275475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.275463    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-maas-api-tls\") pod \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\" (UID: \"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa\") "
Sep 10 21:24:51.275746 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.275719    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" (UID: "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:51.275844 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.275827    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:51.277604 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.277580    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" (UID: "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:51.277891 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.277870    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" (UID: "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:24:51.277958 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.277900    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-kube-api-access-cdmk7" (OuterVolumeSpecName: "kube-api-access-cdmk7") pod "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" (UID: "1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa"). InnerVolumeSpecName "kube-api-access-cdmk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:51.303802 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.303730    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:51.377114 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.377080    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:51.377114 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.377111    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:51.377114 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.377125    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdmk7\" (UniqueName: \"kubernetes.io/projected/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa-kube-api-access-cdmk7\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:24:51.626014 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:51.625913    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:52.219931 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:52.219900    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"
Sep 10 21:24:52.219931 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:52.219912    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn" event={"ID":"1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa","Type":"ContainerDied","Data":"b38c40aa42ea25600400008ecc3e4abc47ea461c140030025b7b42681db4a195"}
Sep 10 21:24:52.220495 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:52.219984    2584 scope.go:117] "RemoveContainer" containerID="b0879fea7179d447299ee131b0256e4e4a1be7a90b2e4d35bbce748aab05ad1c"
Sep 10 21:24:52.243315 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:52.243287    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"]
Sep 10 21:24:52.247585 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:52.247552    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-84b69f-d65fcfd7b-8q9xn"]
Sep 10 21:24:52.301496 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:52.301463    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" path="/var/lib/kubelet/pods/1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa/volumes"
Sep 10 21:24:56.704169 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:24:56.704124    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:34.057682 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:34.057642    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:34.712525 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:34.712475    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:35.425380 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:35.425336    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.067676 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:41.067631    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.168886 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:41.168853    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.348648 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:41.348557    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.813760 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:41.813713    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.194205 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.194167    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"]
Sep 10 21:25:52.194701 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.194474    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5" podUID="8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" containerName="maas-api" containerID="cri-o://e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83" gracePeriod=30
Sep 10 21:25:52.224177 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.224148    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.253317 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.253283    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"]
Sep 10 21:25:52.253565 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.253526    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw" podUID="d781d605-879a-4065-8d79-f754bd2e25f8" containerName="payload-pre-processing" containerID="cri-o://701b5e079863fa11a2e65f3b24f3a39bad224073289d7c4f59cf6aad62ab685e" gracePeriod=30
Sep 10 21:25:52.324042 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.324003    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.428472 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.428230    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.495950 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.495743    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.514368 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.514340    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:25:52.547198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.547162    2584 generic.go:356] "Generic (PLEG): container finished" podID="8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" containerID="e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83" exitCode=0
Sep 10 21:25:52.547327 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.547206    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"
Sep 10 21:25:52.547327 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.547242    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5" event={"ID":"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c","Type":"ContainerDied","Data":"e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83"}
Sep 10 21:25:52.547327 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.547303    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5" event={"ID":"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c","Type":"ContainerDied","Data":"1dc91f3356252f31ff8210e20c83547f806861b2f33eb65097cc77a2c12e9236"}
Sep 10 21:25:52.547533 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.547335    2584 scope.go:117] "RemoveContainer" containerID="e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83"
Sep 10 21:25:52.548799 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.548778    2584 generic.go:356] "Generic (PLEG): container finished" podID="d781d605-879a-4065-8d79-f754bd2e25f8" containerID="701b5e079863fa11a2e65f3b24f3a39bad224073289d7c4f59cf6aad62ab685e" exitCode=0
Sep 10 21:25:52.548874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.548828    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw" event={"ID":"d781d605-879a-4065-8d79-f754bd2e25f8","Type":"ContainerDied","Data":"701b5e079863fa11a2e65f3b24f3a39bad224073289d7c4f59cf6aad62ab685e"}
Sep 10 21:25:52.548874 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.548867    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw" event={"ID":"d781d605-879a-4065-8d79-f754bd2e25f8","Type":"ContainerDied","Data":"39846080c0b742b19abc8b87c781026ed1f154f19f7882cd7b81742f4b56b53b"}
Sep 10 21:25:52.548946 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.548885    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39846080c0b742b19abc8b87c781026ed1f154f19f7882cd7b81742f4b56b53b"
Sep 10 21:25:52.554332 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.554313    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:25:52.557277 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.557257    2584 scope.go:117] "RemoveContainer" containerID="e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83"
Sep 10 21:25:52.557588 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:25:52.557569    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83\": container with ID starting with e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83 not found: ID does not exist" containerID="e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83"
Sep 10 21:25:52.557680 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.557593    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83"} err="failed to get container status \"e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83\": rpc error: code = NotFound desc = could not find container \"e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83\": container with ID starting with e3e8c6182aa5424baccc5b5c3ec4d838183e351f5dfc29e9c561c47676a36c83 not found: ID does not exist"
Sep 10 21:25:52.569908 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.569887    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-metrics-certs\") pod \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") "
Sep 10 21:25:52.569990 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.569944    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-maas-api-tls\") pod \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") "
Sep 10 21:25:52.569990 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.569986    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rvf8\" (UniqueName: \"kubernetes.io/projected/d781d605-879a-4065-8d79-f754bd2e25f8-kube-api-access-4rvf8\") pod \"d781d605-879a-4065-8d79-f754bd2e25f8\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") "
Sep 10 21:25:52.570079 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.570019    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-openshift-service-ca\") pod \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") "
Sep 10 21:25:52.570079 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.570054    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d781d605-879a-4065-8d79-f754bd2e25f8-plugins-config-volume\") pod \"d781d605-879a-4065-8d79-f754bd2e25f8\" (UID: \"d781d605-879a-4065-8d79-f754bd2e25f8\") "
Sep 10 21:25:52.570182 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.570140    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgn9g\" (UniqueName: \"kubernetes.io/projected/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-kube-api-access-wgn9g\") pod \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\" (UID: \"8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c\") "
Sep 10 21:25:52.570400 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.570376    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" (UID: "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:52.570593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.570511    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.570593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.570508    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d781d605-879a-4065-8d79-f754bd2e25f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d781d605-879a-4065-8d79-f754bd2e25f8" (UID: "d781d605-879a-4065-8d79-f754bd2e25f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:52.574436 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.573739    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d781d605-879a-4065-8d79-f754bd2e25f8-kube-api-access-4rvf8" (OuterVolumeSpecName: "kube-api-access-4rvf8") pod "d781d605-879a-4065-8d79-f754bd2e25f8" (UID: "d781d605-879a-4065-8d79-f754bd2e25f8"). InnerVolumeSpecName "kube-api-access-4rvf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:52.574436 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.574166    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-kube-api-access-wgn9g" (OuterVolumeSpecName: "kube-api-access-wgn9g") pod "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" (UID: "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c"). InnerVolumeSpecName "kube-api-access-wgn9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:52.574721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.574684    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" (UID: "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:52.574721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.574709    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" (UID: "8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:52.671161 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.671116    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.671161 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.671156    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.671405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.671171    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rvf8\" (UniqueName: \"kubernetes.io/projected/d781d605-879a-4065-8d79-f754bd2e25f8-kube-api-access-4rvf8\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.671405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.671189    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d781d605-879a-4065-8d79-f754bd2e25f8-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.671405 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.671204    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgn9g\" (UniqueName: \"kubernetes.io/projected/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c-kube-api-access-wgn9g\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.811833 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.811741    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.870205 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.870171    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"]
Sep 10 21:25:52.875594 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:52.875548    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-868994-7d679f6dd-mkrj5"]
Sep 10 21:25:53.530254 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:53.530173    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw" podUID="d781d605-879a-4065-8d79-f754bd2e25f8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.71:9004: i/o timeout"
Sep 10 21:25:53.559095 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:53.559071    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"
Sep 10 21:25:53.581102 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:53.581070    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"]
Sep 10 21:25:53.583976 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:53.583949    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-868994-5bc7c96c54-nz9gw"]
Sep 10 21:25:54.301611 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:54.301575    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" path="/var/lib/kubelet/pods/8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c/volumes"
Sep 10 21:25:54.301972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:54.301958    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d781d605-879a-4065-8d79-f754bd2e25f8" path="/var/lib/kubelet/pods/d781d605-879a-4065-8d79-f754bd2e25f8/volumes"
Sep 10 21:25:59.435721 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:59.435680    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:59.535671 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:59.535631    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:59.744801 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:25:59.744708    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:00.383852 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:00.383816    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:03.526813 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:03.526682    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:08.134319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:08.134281    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:10.261382 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:10.261341    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:10.986239 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:10.986203    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"]
Sep 10 21:26:10.986485 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:10.986459    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt" podUID="fb022a71-1c63-4622-af8a-2bd0a42eddec" containerName="maas-api" containerID="cri-o://0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5" gracePeriod=30
Sep 10 21:26:11.031696 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.031647    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.086236 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.086205    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"]
Sep 10 21:26:11.086485 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.086460    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7" podUID="66b1c2bb-15b5-4933-8dd4-4a7406ace11e" containerName="payload-pre-processing" containerID="cri-o://7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701" gracePeriod=30
Sep 10 21:26:11.141351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.141315    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.254806 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.254739    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:26:11.262323 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.262291    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.334299 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.334262    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.335173 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.335150    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wglcr\" (UniqueName: \"kubernetes.io/projected/fb022a71-1c63-4622-af8a-2bd0a42eddec-kube-api-access-wglcr\") pod \"fb022a71-1c63-4622-af8a-2bd0a42eddec\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") "
Sep 10 21:26:11.335292 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.335275    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-metrics-certs\") pod \"fb022a71-1c63-4622-af8a-2bd0a42eddec\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") "
Sep 10 21:26:11.335390 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.335376    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb022a71-1c63-4622-af8a-2bd0a42eddec-openshift-service-ca\") pod \"fb022a71-1c63-4622-af8a-2bd0a42eddec\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") "
Sep 10 21:26:11.335472 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.335444    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-maas-api-tls\") pod \"fb022a71-1c63-4622-af8a-2bd0a42eddec\" (UID: \"fb022a71-1c63-4622-af8a-2bd0a42eddec\") "
Sep 10 21:26:11.335766 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.335739    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb022a71-1c63-4622-af8a-2bd0a42eddec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb022a71-1c63-4622-af8a-2bd0a42eddec" (UID: "fb022a71-1c63-4622-af8a-2bd0a42eddec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:11.337761 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.337724    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fb022a71-1c63-4622-af8a-2bd0a42eddec" (UID: "fb022a71-1c63-4622-af8a-2bd0a42eddec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:11.337869 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.337787    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb022a71-1c63-4622-af8a-2bd0a42eddec" (UID: "fb022a71-1c63-4622-af8a-2bd0a42eddec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:11.337869 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.337803    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb022a71-1c63-4622-af8a-2bd0a42eddec-kube-api-access-wglcr" (OuterVolumeSpecName: "kube-api-access-wglcr") pod "fb022a71-1c63-4622-af8a-2bd0a42eddec" (UID: "fb022a71-1c63-4622-af8a-2bd0a42eddec"). InnerVolumeSpecName "kube-api-access-wglcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:11.355115 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.355089    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:26:11.436953 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.436921    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st6cp\" (UniqueName: \"kubernetes.io/projected/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-kube-api-access-st6cp\") pod \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") "
Sep 10 21:26:11.437127 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.436981    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-plugins-config-volume\") pod \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\" (UID: \"66b1c2bb-15b5-4933-8dd4-4a7406ace11e\") "
Sep 10 21:26:11.437174 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.437156    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.437174 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.437166    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb022a71-1c63-4622-af8a-2bd0a42eddec-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.437246 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.437175    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb022a71-1c63-4622-af8a-2bd0a42eddec-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.437246 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.437185    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wglcr\" (UniqueName: \"kubernetes.io/projected/fb022a71-1c63-4622-af8a-2bd0a42eddec-kube-api-access-wglcr\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.437369 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.437343    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66b1c2bb-15b5-4933-8dd4-4a7406ace11e" (UID: "66b1c2bb-15b5-4933-8dd4-4a7406ace11e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:11.438926 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.438908    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-kube-api-access-st6cp" (OuterVolumeSpecName: "kube-api-access-st6cp") pod "66b1c2bb-15b5-4933-8dd4-4a7406ace11e" (UID: "66b1c2bb-15b5-4933-8dd4-4a7406ace11e"). InnerVolumeSpecName "kube-api-access-st6cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:11.537900 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.537807    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.537900 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.537840    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st6cp\" (UniqueName: \"kubernetes.io/projected/66b1c2bb-15b5-4933-8dd4-4a7406ace11e-kube-api-access-st6cp\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.652343 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.652305    2584 generic.go:356] "Generic (PLEG): container finished" podID="66b1c2bb-15b5-4933-8dd4-4a7406ace11e" containerID="7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701" exitCode=0
Sep 10 21:26:11.652536 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.652351    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"
Sep 10 21:26:11.652536 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.652399    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7" event={"ID":"66b1c2bb-15b5-4933-8dd4-4a7406ace11e","Type":"ContainerDied","Data":"7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701"}
Sep 10 21:26:11.652536 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.652485    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7" event={"ID":"66b1c2bb-15b5-4933-8dd4-4a7406ace11e","Type":"ContainerDied","Data":"da6d778427ab8cdf20f0a516292328dcb1cc3ba3790ad467c49a6faa548aae0e"}
Sep 10 21:26:11.652536 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.652520    2584 scope.go:117] "RemoveContainer" containerID="7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701"
Sep 10 21:26:11.654039 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.654019    2584 generic.go:356] "Generic (PLEG): container finished" podID="fb022a71-1c63-4622-af8a-2bd0a42eddec" containerID="0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5" exitCode=0
Sep 10 21:26:11.654141 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.654066    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"
Sep 10 21:26:11.654141 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.654107    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt" event={"ID":"fb022a71-1c63-4622-af8a-2bd0a42eddec","Type":"ContainerDied","Data":"0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5"}
Sep 10 21:26:11.654243 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.654160    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt" event={"ID":"fb022a71-1c63-4622-af8a-2bd0a42eddec","Type":"ContainerDied","Data":"194845290f9caa8af7510ed46dc627c79047daea95451b05f9f7ee959bf269f5"}
Sep 10 21:26:11.661860 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.661843    2584 scope.go:117] "RemoveContainer" containerID="7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701"
Sep 10 21:26:11.662192 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:26:11.662166    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701\": container with ID starting with 7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701 not found: ID does not exist" containerID="7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701"
Sep 10 21:26:11.662273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.662203    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701"} err="failed to get container status \"7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701\": rpc error: code = NotFound desc = could not find container \"7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701\": container with ID starting with 7169014c60dcfd0bd2acb7734d0bdfddfc6c83b7eb1de845804afb94466eb701 not found: ID does not exist"
Sep 10 21:26:11.662273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.662229    2584 scope.go:117] "RemoveContainer" containerID="0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5"
Sep 10 21:26:11.670406 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.670389    2584 scope.go:117] "RemoveContainer" containerID="0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5"
Sep 10 21:26:11.670776 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:26:11.670750    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5\": container with ID starting with 0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5 not found: ID does not exist" containerID="0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5"
Sep 10 21:26:11.670872 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.670785    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5"} err="failed to get container status \"0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5\": rpc error: code = NotFound desc = could not find container \"0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5\": container with ID starting with 0b9e3bfe96605f7f2f46c9eb3d3ee85d2760d312bd24c3d5ae4e36108fca80e5 not found: ID does not exist"
Sep 10 21:26:11.680642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.680617    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"]
Sep 10 21:26:11.684927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.684900    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.685252 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.685225    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14acb0-84bd4b9869-tqgzt"]
Sep 10 21:26:11.697984 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.697959    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"]
Sep 10 21:26:11.703147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:11.703126    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14acb0-7796d487c8-tgwd7"]
Sep 10 21:26:12.301035 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:12.300946    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66b1c2bb-15b5-4933-8dd4-4a7406ace11e" path="/var/lib/kubelet/pods/66b1c2bb-15b5-4933-8dd4-4a7406ace11e/volumes"
Sep 10 21:26:12.301412 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:12.301348    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb022a71-1c63-4622-af8a-2bd0a42eddec" path="/var/lib/kubelet/pods/fb022a71-1c63-4622-af8a-2bd0a42eddec/volumes"
Sep 10 21:26:13.086081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:13.086040    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.146002 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:13.145971    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:17.882818 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:17.882777    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:17.984333 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:17.984297    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:18.171141 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:18.171106    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"]
Sep 10 21:26:18.171464 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:18.171403    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy" containerID="cri-o://014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3" gracePeriod=30
Sep 10 21:26:18.186635 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:18.186607    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:18.618731 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:18.618646    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:23.424024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.423996    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:26:23.549105 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549016    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-credential-socket\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549105 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549061    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istiod-ca-cert\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549105 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549094    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdqnl\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-kube-api-access-gdqnl\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549155    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-certs\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549191    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-token\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549217    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-envoy\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549238    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-socket\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549284    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:23.549385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549325    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-data\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549385 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549382    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-podinfo\") pod \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\" (UID: \"e7fdb282-b518-41ab-b67a-75bf7c41ec27\") "
Sep 10 21:26:23.549775 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549498    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:23.549775 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549534    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:23.549775 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549675    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:23.549775 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549693    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:23.549957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549852    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istiod-ca-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.549957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549872    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.549957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549884    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-workload-socket\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.549957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549898    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-data\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.549957 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.549911    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-credential-socket\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.551453 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.551403    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:26:23.551567 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.551495    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:23.551711 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.551695    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-kube-api-access-gdqnl" (OuterVolumeSpecName: "kube-api-access-gdqnl") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "kube-api-access-gdqnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:23.551765 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.551743    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7fdb282-b518-41ab-b67a-75bf7c41ec27" (UID: "e7fdb282-b518-41ab-b67a-75bf7c41ec27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:26:23.651322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.651283    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdqnl\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-kube-api-access-gdqnl\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.651322 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.651316    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-token\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.651547 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.651329    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-envoy\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.651547 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.651341    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7fdb282-b518-41ab-b67a-75bf7c41ec27-istio-podinfo\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:23.729198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.729165    2584 generic.go:356] "Generic (PLEG): container finished" podID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerID="014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3" exitCode=0
Sep 10 21:26:23.729388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.729213    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"
Sep 10 21:26:23.729388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.729250    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" event={"ID":"e7fdb282-b518-41ab-b67a-75bf7c41ec27","Type":"ContainerDied","Data":"014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3"}
Sep 10 21:26:23.729388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.729311    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp" event={"ID":"e7fdb282-b518-41ab-b67a-75bf7c41ec27","Type":"ContainerDied","Data":"e72c489d05bf9846090bb83ac30c27d4e0f93183aaf08b24a596c2e12e97fcd7"}
Sep 10 21:26:23.729388 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.729343    2584 scope.go:117] "RemoveContainer" containerID="014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3"
Sep 10 21:26:23.740709 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.740691    2584 scope.go:117] "RemoveContainer" containerID="a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0"
Sep 10 21:26:23.748820 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.748802    2584 scope.go:117] "RemoveContainer" containerID="014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3"
Sep 10 21:26:23.749081 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:26:23.749057    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3\": container with ID starting with 014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3 not found: ID does not exist" containerID="014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3"
Sep 10 21:26:23.749180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.749086    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3"} err="failed to get container status \"014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3\": rpc error: code = NotFound desc = could not find container \"014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3\": container with ID starting with 014fe99f2dff47b5fd43279829c17c23e8c623c24be47797e10c1d68d0634ae3 not found: ID does not exist"
Sep 10 21:26:23.749180 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.749104    2584 scope.go:117] "RemoveContainer" containerID="a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0"
Sep 10 21:26:23.749387 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:26:23.749361    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0\": container with ID starting with a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0 not found: ID does not exist" containerID="a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0"
Sep 10 21:26:23.749459 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.749392    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0"} err="failed to get container status \"a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0\": rpc error: code = NotFound desc = could not find container \"a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0\": container with ID starting with a721b5fab1e7d077999beca69ae4d71b323fac7b4e075b4a33cff3e83cc6c2b0 not found: ID does not exist"
Sep 10 21:26:23.754972 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.754950    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"]
Sep 10 21:26:23.759582 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:23.759553    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-14acb0-openshift-default-d9f8bfc8b-vclkp"]
Sep 10 21:26:24.307729 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:24.307688    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" path="/var/lib/kubelet/pods/e7fdb282-b518-41ab-b67a-75bf7c41ec27/volumes"
Sep 10 21:26:26.442173 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:26.442139    2584 scope.go:117] "RemoveContainer" containerID="041d3234dcd3c9355ffb22e109f823ec67a447f7227bba1735b0d8b4215016b7"
Sep 10 21:26:29.045203 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.045167    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"]
Sep 10 21:26:29.045719 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.045394    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n" podUID="62785e98-9236-4751-b0c0-488a7bf5ca79" containerName="maas-api" containerID="cri-o://3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347" gracePeriod=30
Sep 10 21:26:29.079147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.079117    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:29.115852 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.115817    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"]
Sep 10 21:26:29.116081 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.116052    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5" podUID="41dcd303-fd05-480f-be8e-7de40e96f5b8" containerName="payload-pre-processing" containerID="cri-o://0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac" gracePeriod=30
Sep 10 21:26:29.185535 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.185494    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:29.283391 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.283321    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:29.355724 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.355646    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:29.370670 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.370644    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:26:29.397321 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.397278    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:26:29.507666 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.507632    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-metrics-certs\") pod \"62785e98-9236-4751-b0c0-488a7bf5ca79\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") "
Sep 10 21:26:29.507812 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.507696    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb79j\" (UniqueName: \"kubernetes.io/projected/41dcd303-fd05-480f-be8e-7de40e96f5b8-kube-api-access-zb79j\") pod \"41dcd303-fd05-480f-be8e-7de40e96f5b8\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") "
Sep 10 21:26:29.507812 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.507743    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88f4j\" (UniqueName: \"kubernetes.io/projected/62785e98-9236-4751-b0c0-488a7bf5ca79-kube-api-access-88f4j\") pod \"62785e98-9236-4751-b0c0-488a7bf5ca79\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") "
Sep 10 21:26:29.507812 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.507788    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62785e98-9236-4751-b0c0-488a7bf5ca79-openshift-service-ca\") pod \"62785e98-9236-4751-b0c0-488a7bf5ca79\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") "
Sep 10 21:26:29.507990 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.507821    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls\") pod \"62785e98-9236-4751-b0c0-488a7bf5ca79\" (UID: \"62785e98-9236-4751-b0c0-488a7bf5ca79\") "
Sep 10 21:26:29.507990 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.507846    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41dcd303-fd05-480f-be8e-7de40e96f5b8-plugins-config-volume\") pod \"41dcd303-fd05-480f-be8e-7de40e96f5b8\" (UID: \"41dcd303-fd05-480f-be8e-7de40e96f5b8\") "
Sep 10 21:26:29.508226 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.508195    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62785e98-9236-4751-b0c0-488a7bf5ca79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62785e98-9236-4751-b0c0-488a7bf5ca79" (UID: "62785e98-9236-4751-b0c0-488a7bf5ca79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:29.508339 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.508290    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41dcd303-fd05-480f-be8e-7de40e96f5b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41dcd303-fd05-480f-be8e-7de40e96f5b8" (UID: "41dcd303-fd05-480f-be8e-7de40e96f5b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:29.509993 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.509966    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62785e98-9236-4751-b0c0-488a7bf5ca79-kube-api-access-88f4j" (OuterVolumeSpecName: "kube-api-access-88f4j") pod "62785e98-9236-4751-b0c0-488a7bf5ca79" (UID: "62785e98-9236-4751-b0c0-488a7bf5ca79"). InnerVolumeSpecName "kube-api-access-88f4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:29.510103 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.510012    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "62785e98-9236-4751-b0c0-488a7bf5ca79" (UID: "62785e98-9236-4751-b0c0-488a7bf5ca79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:29.510103 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.510024    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62785e98-9236-4751-b0c0-488a7bf5ca79" (UID: "62785e98-9236-4751-b0c0-488a7bf5ca79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:29.510103 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.510028    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41dcd303-fd05-480f-be8e-7de40e96f5b8-kube-api-access-zb79j" (OuterVolumeSpecName: "kube-api-access-zb79j") pod "41dcd303-fd05-480f-be8e-7de40e96f5b8" (UID: "41dcd303-fd05-480f-be8e-7de40e96f5b8"). InnerVolumeSpecName "kube-api-access-zb79j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:29.608808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.608737    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88f4j\" (UniqueName: \"kubernetes.io/projected/62785e98-9236-4751-b0c0-488a7bf5ca79-kube-api-access-88f4j\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.608808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.608761    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62785e98-9236-4751-b0c0-488a7bf5ca79-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.608808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.608771    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.608808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.608783    2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41dcd303-fd05-480f-be8e-7de40e96f5b8-plugins-config-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.608808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.608792    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/62785e98-9236-4751-b0c0-488a7bf5ca79-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.608808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.608800    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb79j\" (UniqueName: \"kubernetes.io/projected/41dcd303-fd05-480f-be8e-7de40e96f5b8-kube-api-access-zb79j\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.686337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.686301    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:29.763997 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.763957    2584 generic.go:356] "Generic (PLEG): container finished" podID="62785e98-9236-4751-b0c0-488a7bf5ca79" containerID="3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347" exitCode=0
Sep 10 21:26:29.764184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.764013    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"
Sep 10 21:26:29.764184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.764037    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n" event={"ID":"62785e98-9236-4751-b0c0-488a7bf5ca79","Type":"ContainerDied","Data":"3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347"}
Sep 10 21:26:29.764184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.764127    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n" event={"ID":"62785e98-9236-4751-b0c0-488a7bf5ca79","Type":"ContainerDied","Data":"4aaa68da95d4663f27ae7e22d3caf56c769dacc6f60b3b1990a8a2bc1a06b820"}
Sep 10 21:26:29.764184 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.764161    2584 scope.go:117] "RemoveContainer" containerID="3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347"
Sep 10 21:26:29.765862 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.765830    2584 generic.go:356] "Generic (PLEG): container finished" podID="41dcd303-fd05-480f-be8e-7de40e96f5b8" containerID="0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac" exitCode=0
Sep 10 21:26:29.765986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.765878    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5" event={"ID":"41dcd303-fd05-480f-be8e-7de40e96f5b8","Type":"ContainerDied","Data":"0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac"}
Sep 10 21:26:29.765986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.765920    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"
Sep 10 21:26:29.765986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.765969    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5" event={"ID":"41dcd303-fd05-480f-be8e-7de40e96f5b8","Type":"ContainerDied","Data":"86412ea22177852a5dcab00f518d57d0e28a52321b2c06547582f48f0a56997a"}
Sep 10 21:26:29.775120 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.775102    2584 scope.go:117] "RemoveContainer" containerID="3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347"
Sep 10 21:26:29.775401 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:26:29.775384    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347\": container with ID starting with 3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347 not found: ID does not exist" containerID="3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347"
Sep 10 21:26:29.775490 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.775408    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347"} err="failed to get container status \"3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347\": rpc error: code = NotFound desc = could not find container \"3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347\": container with ID starting with 3f3e0b6fba21ff44227c057ca5bbdca710ea73751533163e6f170e48ef079347 not found: ID does not exist"
Sep 10 21:26:29.775490 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.775451    2584 scope.go:117] "RemoveContainer" containerID="0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac"
Sep 10 21:26:29.783186 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.783166    2584 scope.go:117] "RemoveContainer" containerID="0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac"
Sep 10 21:26:29.783500 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:26:29.783482    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac\": container with ID starting with 0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac not found: ID does not exist" containerID="0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac"
Sep 10 21:26:29.783561 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.783507    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac"} err="failed to get container status \"0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac\": rpc error: code = NotFound desc = could not find container \"0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac\": container with ID starting with 0bfb9995559fe1a5e44c44e0374120572f6c90ddd008c75d0d00fc5db78e19ac not found: ID does not exist"
Sep 10 21:26:29.790564 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.790538    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"]
Sep 10 21:26:29.796452 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.796407    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-755926-6b4b47466f-jlm7n"]
Sep 10 21:26:29.810891 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.810865    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"]
Sep 10 21:26:29.814277 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:29.814256    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-755926-6fff89764b-wqfz5"]
Sep 10 21:26:30.301303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:30.301269    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41dcd303-fd05-480f-be8e-7de40e96f5b8" path="/var/lib/kubelet/pods/41dcd303-fd05-480f-be8e-7de40e96f5b8/volumes"
Sep 10 21:26:30.301717 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:30.301702    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62785e98-9236-4751-b0c0-488a7bf5ca79" path="/var/lib/kubelet/pods/62785e98-9236-4751-b0c0-488a7bf5ca79/volumes"
Sep 10 21:26:36.568168 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:36.568128    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:36.665181 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:36.665140    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:36.803320 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:36.803288    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:39.269575 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.269541    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-wp48c"]
Sep 10 21:26:39.270018 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.269746    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76f4d89c9c-wp48c" podUID="6f53f698-2d72-4f15-8313-38ee72e1e4d1" containerName="manager" containerID="cri-o://a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb" gracePeriod=30
Sep 10 21:26:39.517558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.517534    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:26:39.583294 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.583204    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert\") pod \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") "
Sep 10 21:26:39.583558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.583312    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk824\" (UniqueName: \"kubernetes.io/projected/6f53f698-2d72-4f15-8313-38ee72e1e4d1-kube-api-access-tk824\") pod \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") "
Sep 10 21:26:39.583558 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.583357    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs\") pod \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\" (UID: \"6f53f698-2d72-4f15-8313-38ee72e1e4d1\") "
Sep 10 21:26:39.585883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.585762    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f53f698-2d72-4f15-8313-38ee72e1e4d1" (UID: "6f53f698-2d72-4f15-8313-38ee72e1e4d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:39.585883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.585837    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6f53f698-2d72-4f15-8313-38ee72e1e4d1" (UID: "6f53f698-2d72-4f15-8313-38ee72e1e4d1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:39.585883 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.585851    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f53f698-2d72-4f15-8313-38ee72e1e4d1-kube-api-access-tk824" (OuterVolumeSpecName: "kube-api-access-tk824") pod "6f53f698-2d72-4f15-8313-38ee72e1e4d1" (UID: "6f53f698-2d72-4f15-8313-38ee72e1e4d1"). InnerVolumeSpecName "kube-api-access-tk824". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:39.685138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.685105    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tk824\" (UniqueName: \"kubernetes.io/projected/6f53f698-2d72-4f15-8313-38ee72e1e4d1-kube-api-access-tk824\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:39.685138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.685132    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:39.685138 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.685141    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f53f698-2d72-4f15-8313-38ee72e1e4d1-webhook-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:26:39.820726 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.820690    2584 generic.go:356] "Generic (PLEG): container finished" podID="6f53f698-2d72-4f15-8313-38ee72e1e4d1" containerID="a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb" exitCode=0
Sep 10 21:26:39.820937 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.820737    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-wp48c"
Sep 10 21:26:39.820937 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.820745    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-wp48c" event={"ID":"6f53f698-2d72-4f15-8313-38ee72e1e4d1","Type":"ContainerDied","Data":"a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb"}
Sep 10 21:26:39.820937 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.820801    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-wp48c" event={"ID":"6f53f698-2d72-4f15-8313-38ee72e1e4d1","Type":"ContainerDied","Data":"185c367f7daeda4b9f71b84d3f39a5f75835ba1e3a321d6c300029311b625fef"}
Sep 10 21:26:39.820937 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.820830    2584 scope.go:117] "RemoveContainer" containerID="a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb"
Sep 10 21:26:39.830462 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.830444    2584 scope.go:117] "RemoveContainer" containerID="a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb"
Sep 10 21:26:39.830758 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:26:39.830736    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb\": container with ID starting with a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb not found: ID does not exist" containerID="a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb"
Sep 10 21:26:39.830809 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.830767    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb"} err="failed to get container status \"a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb\": rpc error: code = NotFound desc = could not find container \"a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb\": container with ID starting with a3033a3079cf82def8ad3c523f6a3cb0fe497bd5f157176368a13b5990ff8adb not found: ID does not exist"
Sep 10 21:26:39.844863 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.844796    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-wp48c"]
Sep 10 21:26:39.851942 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:39.851919    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-wp48c"]
Sep 10 21:26:40.301140 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:40.301103    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f53f698-2d72-4f15-8313-38ee72e1e4d1" path="/var/lib/kubelet/pods/6f53f698-2d72-4f15-8313-38ee72e1e4d1/volumes"
Sep 10 21:26:43.646725 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:26:43.646690    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:40.702377 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702338    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-gdb5m"]
Sep 10 21:27:40.702927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702903    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1298515e-e196-4b00-bb34-29601d4b25d0" containerName="maas-api"
Sep 10 21:27:40.702927 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702925    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1298515e-e196-4b00-bb34-29601d4b25d0" containerName="maas-api"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702939    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d781d605-879a-4065-8d79-f754bd2e25f8" containerName="payload-pre-processing"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702947    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d781d605-879a-4065-8d79-f754bd2e25f8" containerName="payload-pre-processing"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702961    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66b1c2bb-15b5-4933-8dd4-4a7406ace11e" containerName="payload-pre-processing"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702970    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="66b1c2bb-15b5-4933-8dd4-4a7406ace11e" containerName="payload-pre-processing"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702988    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f53f698-2d72-4f15-8313-38ee72e1e4d1" containerName="manager"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.702996    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f53f698-2d72-4f15-8313-38ee72e1e4d1" containerName="manager"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703007    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62785e98-9236-4751-b0c0-488a7bf5ca79" containerName="maas-api"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703015    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="62785e98-9236-4751-b0c0-488a7bf5ca79" containerName="maas-api"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703026    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703034    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703047    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703055    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy"
Sep 10 21:27:40.703059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703064    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41dcd303-fd05-480f-be8e-7de40e96f5b8" containerName="payload-pre-processing"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703072    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="41dcd303-fd05-480f-be8e-7de40e96f5b8" containerName="payload-pre-processing"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703083    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb022a71-1c63-4622-af8a-2bd0a42eddec" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703092    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb022a71-1c63-4622-af8a-2bd0a42eddec" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703110    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703120    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703138    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703147    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703233    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="62785e98-9236-4751-b0c0-488a7bf5ca79" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703245    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dbc7f38-2281-4a0b-a9d5-1ad3f999aa3c" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703253    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="41dcd303-fd05-480f-be8e-7de40e96f5b8" containerName="payload-pre-processing"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703262    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d93f2db-90df-4cdc-9b4e-3bbe4694fdaa" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703271    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1298515e-e196-4b00-bb34-29601d4b25d0" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703282    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb022a71-1c63-4622-af8a-2bd0a42eddec" containerName="maas-api"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703290    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="66b1c2bb-15b5-4933-8dd4-4a7406ace11e" containerName="payload-pre-processing"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703297    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d781d605-879a-4065-8d79-f754bd2e25f8" containerName="payload-pre-processing"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703306    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f53f698-2d72-4f15-8313-38ee72e1e4d1" containerName="manager"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703315    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy"
Sep 10 21:27:40.703740 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.703323    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7fdb282-b518-41ab-b67a-75bf7c41ec27" containerName="istio-proxy"
Sep 10 21:27:40.706628 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.706608    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.712072 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.712037    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 21:27:40.712192 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.712175    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ddvx9\""
Sep 10 21:27:40.713215 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.712840    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 21:27:40.713319 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.713303    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 21:27:40.733466 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.733441    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-gdb5m"]
Sep 10 21:27:40.816868 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.816834    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-webhook-cert\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.817027 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.816882    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-metrics-certs\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.817027 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.816903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx6d4\" (UniqueName: \"kubernetes.io/projected/55cf194e-833c-4c78-bc9c-d22b78bbb45c-kube-api-access-sx6d4\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.918212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.918175    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-metrics-certs\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.918212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.918214    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx6d4\" (UniqueName: \"kubernetes.io/projected/55cf194e-833c-4c78-bc9c-d22b78bbb45c-kube-api-access-sx6d4\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.918443 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.918273    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-webhook-cert\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.920714 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.920684    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-metrics-certs\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.920830 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.920761    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-webhook-cert\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:40.931241 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:40.931219    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx6d4\" (UniqueName: \"kubernetes.io/projected/55cf194e-833c-4c78-bc9c-d22b78bbb45c-kube-api-access-sx6d4\") pod \"maas-controller-76f4d89c9c-gdb5m\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") " pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:41.016711 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:41.016633    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:41.308690 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:41.308608    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-gdb5m"]
Sep 10 21:27:41.313979 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:27:41.313951    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cf194e_833c_4c78_bc9c_d22b78bbb45c.slice/crio-b1326e5b3d3dcc9ace41d5afb7e53db94647aab25b5f8a5e608770e04da97db0 WatchSource:0}: Error finding container b1326e5b3d3dcc9ace41d5afb7e53db94647aab25b5f8a5e608770e04da97db0: Status 404 returned error can't find the container with id b1326e5b3d3dcc9ace41d5afb7e53db94647aab25b5f8a5e608770e04da97db0
Sep 10 21:27:41.315294 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:41.315276    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:27:42.125752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:42.125659    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m" event={"ID":"55cf194e-833c-4c78-bc9c-d22b78bbb45c","Type":"ContainerStarted","Data":"c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e"}
Sep 10 21:27:42.125752 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:42.125713    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m" event={"ID":"55cf194e-833c-4c78-bc9c-d22b78bbb45c","Type":"ContainerStarted","Data":"b1326e5b3d3dcc9ace41d5afb7e53db94647aab25b5f8a5e608770e04da97db0"}
Sep 10 21:27:42.126165 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:42.125778    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:53.132679 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:53.132648    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:27:53.159551 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:27:53.159499    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m" podStartSLOduration=12.636716831 podStartE2EDuration="13.159484925s" totalImagesPullingTime="522.768094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:27:42.164214819 +0000 UTC m=+1756.489975740" watchObservedRunningTime="2026-09-10 21:27:53.159484925 +0000 UTC m=+1767.485245846"
Sep 10 21:28:02.260633 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:02.260598    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:04.463334 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.463298    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:04.549698 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.549662    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-gdb5m"]
Sep 10 21:28:04.549881 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.549859    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m" podUID="55cf194e-833c-4c78-bc9c-d22b78bbb45c" containerName="manager" containerID="cri-o://c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e" gracePeriod=30
Sep 10 21:28:04.795778 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.795752    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:28:04.938085 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.938052    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-metrics-certs\") pod \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") "
Sep 10 21:28:04.938285 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.938115    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-webhook-cert\") pod \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") "
Sep 10 21:28:04.938285 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.938201    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx6d4\" (UniqueName: \"kubernetes.io/projected/55cf194e-833c-4c78-bc9c-d22b78bbb45c-kube-api-access-sx6d4\") pod \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\" (UID: \"55cf194e-833c-4c78-bc9c-d22b78bbb45c\") "
Sep 10 21:28:04.940170 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.940142    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "55cf194e-833c-4c78-bc9c-d22b78bbb45c" (UID: "55cf194e-833c-4c78-bc9c-d22b78bbb45c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:04.940293 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.940187    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55cf194e-833c-4c78-bc9c-d22b78bbb45c" (UID: "55cf194e-833c-4c78-bc9c-d22b78bbb45c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:04.940293 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:04.940273    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55cf194e-833c-4c78-bc9c-d22b78bbb45c-kube-api-access-sx6d4" (OuterVolumeSpecName: "kube-api-access-sx6d4") pod "55cf194e-833c-4c78-bc9c-d22b78bbb45c" (UID: "55cf194e-833c-4c78-bc9c-d22b78bbb45c"). InnerVolumeSpecName "kube-api-access-sx6d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:05.039260 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.039178    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx6d4\" (UniqueName: \"kubernetes.io/projected/55cf194e-833c-4c78-bc9c-d22b78bbb45c-kube-api-access-sx6d4\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.039260 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.039208    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.039260 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.039218    2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/55cf194e-833c-4c78-bc9c-d22b78bbb45c-webhook-cert\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:28:05.234941 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.234906    2584 generic.go:356] "Generic (PLEG): container finished" podID="55cf194e-833c-4c78-bc9c-d22b78bbb45c" containerID="c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e" exitCode=0
Sep 10 21:28:05.235122 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.234946    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m"
Sep 10 21:28:05.235122 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.234969    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m" event={"ID":"55cf194e-833c-4c78-bc9c-d22b78bbb45c","Type":"ContainerDied","Data":"c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e"}
Sep 10 21:28:05.235122 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.235009    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-gdb5m" event={"ID":"55cf194e-833c-4c78-bc9c-d22b78bbb45c","Type":"ContainerDied","Data":"b1326e5b3d3dcc9ace41d5afb7e53db94647aab25b5f8a5e608770e04da97db0"}
Sep 10 21:28:05.235122 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.235030    2584 scope.go:117] "RemoveContainer" containerID="c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e"
Sep 10 21:28:05.244277 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.244259    2584 scope.go:117] "RemoveContainer" containerID="c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e"
Sep 10 21:28:05.244557 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:28:05.244536    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e\": container with ID starting with c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e not found: ID does not exist" containerID="c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e"
Sep 10 21:28:05.244611 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.244565    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e"} err="failed to get container status \"c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e\": rpc error: code = NotFound desc = could not find container \"c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e\": container with ID starting with c1da95a146131ee90b0de12952b489d5621fff9ae2628a1e645bb2bc55d7759e not found: ID does not exist"
Sep 10 21:28:05.257407 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.257381    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-gdb5m"]
Sep 10 21:28:05.261050 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.261028    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-gdb5m"]
Sep 10 21:28:05.917738 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:05.917702    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:06.017994 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.017957    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-5k6m5"]
Sep 10 21:28:06.018410 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.018392    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55cf194e-833c-4c78-bc9c-d22b78bbb45c" containerName="manager"
Sep 10 21:28:06.018410 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.018412    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="55cf194e-833c-4c78-bc9c-d22b78bbb45c" containerName="manager"
Sep 10 21:28:06.018595 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.018509    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="55cf194e-833c-4c78-bc9c-d22b78bbb45c" containerName="manager"
Sep 10 21:28:06.022709 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.022689    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.026365 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.026345    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 21:28:06.026512 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.026380    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 21:28:06.026512 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.026406    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 21:28:06.026796 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.026773    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ddvx9\""
Sep 10 21:28:06.037485 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.037461    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-5k6m5"]
Sep 10 21:28:06.148895 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.148853    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9sfw\" (UniqueName: \"kubernetes.io/projected/36817527-9b94-484a-9dae-01fa69a2d3fe-kube-api-access-t9sfw\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.148895 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.148894    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36817527-9b94-484a-9dae-01fa69a2d3fe-metrics-certs\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.149109 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.149013    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36817527-9b94-484a-9dae-01fa69a2d3fe-webhook-cert\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.250379 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.250293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36817527-9b94-484a-9dae-01fa69a2d3fe-webhook-cert\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.250576 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.250453    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9sfw\" (UniqueName: \"kubernetes.io/projected/36817527-9b94-484a-9dae-01fa69a2d3fe-kube-api-access-t9sfw\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.250576 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.250488    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36817527-9b94-484a-9dae-01fa69a2d3fe-metrics-certs\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.252969 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.252944    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36817527-9b94-484a-9dae-01fa69a2d3fe-metrics-certs\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.253075 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.252990    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36817527-9b94-484a-9dae-01fa69a2d3fe-webhook-cert\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.259692 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.259666    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9sfw\" (UniqueName: \"kubernetes.io/projected/36817527-9b94-484a-9dae-01fa69a2d3fe-kube-api-access-t9sfw\") pod \"maas-controller-76f4d89c9c-5k6m5\" (UID: \"36817527-9b94-484a-9dae-01fa69a2d3fe\") " pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.305760 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.305727    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55cf194e-833c-4c78-bc9c-d22b78bbb45c" path="/var/lib/kubelet/pods/55cf194e-833c-4c78-bc9c-d22b78bbb45c/volumes"
Sep 10 21:28:06.333732 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.333705    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:06.604156 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:06.604112    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f4d89c9c-5k6m5"]
Sep 10 21:28:06.606535 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:28:06.606510    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36817527_9b94_484a_9dae_01fa69a2d3fe.slice/crio-076207f989800bce87c78792976739582ea3688aa2b52ec91f6c06a1d6317006 WatchSource:0}: Error finding container 076207f989800bce87c78792976739582ea3688aa2b52ec91f6c06a1d6317006: Status 404 returned error can't find the container with id 076207f989800bce87c78792976739582ea3688aa2b52ec91f6c06a1d6317006
Sep 10 21:28:07.247099 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:07.247063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-5k6m5" event={"ID":"36817527-9b94-484a-9dae-01fa69a2d3fe","Type":"ContainerStarted","Data":"d7dbe1bac8768592da131d286b3f62e46e4939dd99c6b57958d5abdf9ec22a46"}
Sep 10 21:28:07.247522 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:07.247113    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f4d89c9c-5k6m5" event={"ID":"36817527-9b94-484a-9dae-01fa69a2d3fe","Type":"ContainerStarted","Data":"076207f989800bce87c78792976739582ea3688aa2b52ec91f6c06a1d6317006"}
Sep 10 21:28:07.247522 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:07.247171    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:07.268294 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:07.268240    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f4d89c9c-5k6m5" podStartSLOduration=0.799914146 podStartE2EDuration="1.26822041s" totalImagesPullingTime="468.306264ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:07.26764902 +0000 UTC m=+1781.593409942" watchObservedRunningTime="2026-09-10 21:28:07.26822041 +0000 UTC m=+1781.593981347"
Sep 10 21:28:18.254303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:18.254271    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f4d89c9c-5k6m5"
Sep 10 21:28:26.254232 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:26.254197    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:28:26.256290 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:28:26.256268    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:29:21.698856 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:21.698823    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"]
Sep 10 21:29:21.699443 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:21.699059    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc" podUID="d3903999-961e-4545-bd07-c48ceca8c1d4" containerName="manager" containerID="cri-o://c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4" gracePeriod=10
Sep 10 21:29:21.954159 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:21.954098    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:29:22.020475 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.020445    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nqzn\" (UniqueName: \"kubernetes.io/projected/d3903999-961e-4545-bd07-c48ceca8c1d4-kube-api-access-9nqzn\") pod \"d3903999-961e-4545-bd07-c48ceca8c1d4\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") "
Sep 10 21:29:22.020638 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.020593    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3903999-961e-4545-bd07-c48ceca8c1d4-extensions-socket-volume\") pod \"d3903999-961e-4545-bd07-c48ceca8c1d4\" (UID: \"d3903999-961e-4545-bd07-c48ceca8c1d4\") "
Sep 10 21:29:22.021011 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.020981    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3903999-961e-4545-bd07-c48ceca8c1d4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d3903999-961e-4545-bd07-c48ceca8c1d4" (UID: "d3903999-961e-4545-bd07-c48ceca8c1d4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:29:22.022384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.022348    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3903999-961e-4545-bd07-c48ceca8c1d4-kube-api-access-9nqzn" (OuterVolumeSpecName: "kube-api-access-9nqzn") pod "d3903999-961e-4545-bd07-c48ceca8c1d4" (UID: "d3903999-961e-4545-bd07-c48ceca8c1d4"). InnerVolumeSpecName "kube-api-access-9nqzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:29:22.122198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.122167    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nqzn\" (UniqueName: \"kubernetes.io/projected/d3903999-961e-4545-bd07-c48ceca8c1d4-kube-api-access-9nqzn\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:29:22.122198 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.122194    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3903999-961e-4545-bd07-c48ceca8c1d4-extensions-socket-volume\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:29:22.618452 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.618352    2584 generic.go:356] "Generic (PLEG): container finished" podID="d3903999-961e-4545-bd07-c48ceca8c1d4" containerID="c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4" exitCode=0
Sep 10 21:29:22.618452 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.618395    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"
Sep 10 21:29:22.618452 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.618443    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc" event={"ID":"d3903999-961e-4545-bd07-c48ceca8c1d4","Type":"ContainerDied","Data":"c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4"}
Sep 10 21:29:22.618704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.618505    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc" event={"ID":"d3903999-961e-4545-bd07-c48ceca8c1d4","Type":"ContainerDied","Data":"4ce6a4e927c10316fad6514eeb3dfb3eaa0f302c55cdc52c0f4546a117df6616"}
Sep 10 21:29:22.618704 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.618529    2584 scope.go:117] "RemoveContainer" containerID="c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4"
Sep 10 21:29:22.627662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.627631    2584 scope.go:117] "RemoveContainer" containerID="c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4"
Sep 10 21:29:22.627929 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:29:22.627910    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4\": container with ID starting with c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4 not found: ID does not exist" containerID="c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4"
Sep 10 21:29:22.627994 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.627940    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4"} err="failed to get container status \"c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4\": rpc error: code = NotFound desc = could not find container \"c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4\": container with ID starting with c1fad0c7e3227499f3488bf0715cc04b5276bc03e2fb714f2b102895bd6504e4 not found: ID does not exist"
Sep 10 21:29:22.636992 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.636967    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"]
Sep 10 21:29:22.641359 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:22.641336    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-nlsqc"]
Sep 10 21:29:24.300607 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:24.300571    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3903999-961e-4545-bd07-c48ceca8c1d4" path="/var/lib/kubelet/pods/d3903999-961e-4545-bd07-c48ceca8c1d4/volumes"
Sep 10 21:29:26.502841 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:29:26.502805    2584 scope.go:117] "RemoveContainer" containerID="701b5e079863fa11a2e65f3b24f3a39bad224073289d7c4f59cf6aad62ab685e"
Sep 10 21:30:22.364313 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.364272    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"]
Sep 10 21:30:22.364858 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.364793    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3903999-961e-4545-bd07-c48ceca8c1d4" containerName="manager"
Sep 10 21:30:22.364858 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.364814    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3903999-961e-4545-bd07-c48ceca8c1d4" containerName="manager"
Sep 10 21:30:22.364974 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.364932    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3903999-961e-4545-bd07-c48ceca8c1d4" containerName="manager"
Sep 10 21:30:22.367878 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.367857    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.370170 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.370149    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jjpzr\""
Sep 10 21:30:22.380662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.380638    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"]
Sep 10 21:30:22.441260 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.441225    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwj6z\" (UniqueName: \"kubernetes.io/projected/6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd-kube-api-access-rwj6z\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mhg4d\" (UID: \"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.441440 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.441291    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mhg4d\" (UID: \"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.542197 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.542157    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mhg4d\" (UID: \"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.542351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.542252    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwj6z\" (UniqueName: \"kubernetes.io/projected/6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd-kube-api-access-rwj6z\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mhg4d\" (UID: \"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.542637 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.542616    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mhg4d\" (UID: \"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.550968 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.550936    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwj6z\" (UniqueName: \"kubernetes.io/projected/6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd-kube-api-access-rwj6z\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mhg4d\" (UID: \"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.677851 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.677806    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:22.950636 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:22.950443    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"]
Sep 10 21:30:22.953551 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:30:22.953523    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c88b9f6_6d87_4c9f_a839_51a9ce8dfbcd.slice/crio-d6d5802ad0e4dc787030b9c0c467d493f5e2d3b0449c53e4c376d35ac4dd17c6 WatchSource:0}: Error finding container d6d5802ad0e4dc787030b9c0c467d493f5e2d3b0449c53e4c376d35ac4dd17c6: Status 404 returned error can't find the container with id d6d5802ad0e4dc787030b9c0c467d493f5e2d3b0449c53e4c376d35ac4dd17c6
Sep 10 21:30:23.911258 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:23.911221    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d" event={"ID":"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd","Type":"ContainerStarted","Data":"4b4737ba1d22b1e3960172b83f60641c4754d7a78683e99e991f79ac35565cdb"}
Sep 10 21:30:23.911705 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:23.911275    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d" event={"ID":"6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd","Type":"ContainerStarted","Data":"d6d5802ad0e4dc787030b9c0c467d493f5e2d3b0449c53e4c376d35ac4dd17c6"}
Sep 10 21:30:23.911705 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:23.911327    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:30:23.933942 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:23.933894    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d" podStartSLOduration=1.933879006 podStartE2EDuration="1.933879006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:23.932447234 +0000 UTC m=+1918.258208156" watchObservedRunningTime="2026-09-10 21:30:23.933879006 +0000 UTC m=+1918.259639927"
Sep 10 21:30:34.918130 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:30:34.918098    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mhg4d"
Sep 10 21:31:11.223091 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:11.223041    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:23.416462 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.416346    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"]
Sep 10 21:31:23.420747 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.420719    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.425101 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.425072    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-650a5a7f-kserve-self-signed-certs\""
Sep 10 21:31:23.431873 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.431848    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"]
Sep 10 21:31:23.458044 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.458013    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:23.497845 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.497802    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrq6x\" (UniqueName: \"kubernetes.io/projected/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-kube-api-access-vrq6x\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.498053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.497870    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-dshm\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.498053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.497911    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-model-cache\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.498053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.497962    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-home\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.498053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.498020    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tmp-dir\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.498273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.498072    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tls-certs\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.598808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.598770    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-dshm\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.598808 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.598812    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-model-cache\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.599058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.598847    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-home\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.599058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.598883    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tmp-dir\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.599058 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.599027    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tls-certs\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.599210 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.599114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrq6x\" (UniqueName: \"kubernetes.io/projected/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-kube-api-access-vrq6x\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.599305 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.599286    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-home\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.599351 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.599318    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-model-cache\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.599395 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.599358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tmp-dir\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.601097 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.601075    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-dshm\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.601497 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.601477    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tls-certs\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.621147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.621117    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrq6x\" (UniqueName: \"kubernetes.io/projected/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-kube-api-access-vrq6x\") pod \"e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") " pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:23.733926 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:23.733842    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:24.007514 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:24.007432    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"]
Sep 10 21:31:24.010489 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:31:24.010461    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6ea3f5f_54a1_4687_9916_b6c42f0c3a03.slice/crio-a2eff37a7e5294e8ee086a20e47fabe9b7d1957ae2f60cad6c3d6a74ae39234f WatchSource:0}: Error finding container a2eff37a7e5294e8ee086a20e47fabe9b7d1957ae2f60cad6c3d6a74ae39234f: Status 404 returned error can't find the container with id a2eff37a7e5294e8ee086a20e47fabe9b7d1957ae2f60cad6c3d6a74ae39234f
Sep 10 21:31:24.191253 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:24.191210    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9" event={"ID":"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03","Type":"ContainerStarted","Data":"062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda"}
Sep 10 21:31:24.191479 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:24.191267    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9" event={"ID":"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03","Type":"ContainerStarted","Data":"a2eff37a7e5294e8ee086a20e47fabe9b7d1957ae2f60cad6c3d6a74ae39234f"}
Sep 10 21:31:24.208243 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:24.208184    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9" podStartSLOduration=1.208164134 podStartE2EDuration="1.208164134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:31:24.206995566 +0000 UTC m=+1978.532756486" watchObservedRunningTime="2026-09-10 21:31:24.208164134 +0000 UTC m=+1978.533925056"
Sep 10 21:31:33.734837 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:33.734798    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:33.735255 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:33.734851    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:33.747352 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:33.747319    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:31:34.250045 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:31:34.250011    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:32:06.594658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:06.594623    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:08.387694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:08.387658    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:10.064247 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:10.064207    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"]
Sep 10 21:32:10.089186 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:10.089154    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:25.089330 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.089279    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9" podUID="e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" containerName="main" containerID="cri-o://062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda" gracePeriod=45
Sep 10 21:32:25.342852 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.342794    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:32:25.471134 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471095    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-model-cache\") pod \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") "
Sep 10 21:32:25.471337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471188    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-dshm\") pod \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") "
Sep 10 21:32:25.471337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471220    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tmp-dir\") pod \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") "
Sep 10 21:32:25.471337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471256    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tls-certs\") pod \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") "
Sep 10 21:32:25.471337 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471295    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-home\") pod \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") "
Sep 10 21:32:25.471589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471371    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrq6x\" (UniqueName: \"kubernetes.io/projected/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-kube-api-access-vrq6x\") pod \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\" (UID: \"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03\") "
Sep 10 21:32:25.471589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471406    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" (UID: "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:25.471589 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471472    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" (UID: "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:25.471751 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471611    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-home" (OuterVolumeSpecName: "home") pod "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" (UID: "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:25.471751 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471715    2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-model-cache\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:25.471751 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471733    2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tmp-dir\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:25.471751 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.471742    2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-home\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:25.473644 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.473609    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-kube-api-access-vrq6x" (OuterVolumeSpecName: "kube-api-access-vrq6x") pod "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" (UID: "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03"). InnerVolumeSpecName "kube-api-access-vrq6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:25.473911 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.473884    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-dshm" (OuterVolumeSpecName: "dshm") pod "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" (UID: "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:25.473911 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.473897    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" (UID: "e6ea3f5f-54a1-4687-9916-b6c42f0c3a03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:25.479254 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.479225    2584 generic.go:356] "Generic (PLEG): container finished" podID="e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" containerID="062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda" exitCode=0
Sep 10 21:32:25.479384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.479271    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"
Sep 10 21:32:25.479384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.479303    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9" event={"ID":"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03","Type":"ContainerDied","Data":"062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda"}
Sep 10 21:32:25.479384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.479360    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9" event={"ID":"e6ea3f5f-54a1-4687-9916-b6c42f0c3a03","Type":"ContainerDied","Data":"a2eff37a7e5294e8ee086a20e47fabe9b7d1957ae2f60cad6c3d6a74ae39234f"}
Sep 10 21:32:25.479384 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.479383    2584 scope.go:117] "RemoveContainer" containerID="062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda"
Sep 10 21:32:25.493032 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.493014    2584 scope.go:117] "RemoveContainer" containerID="062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda"
Sep 10 21:32:25.493289 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:32:25.493270    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda\": container with ID starting with 062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda not found: ID does not exist" containerID="062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda"
Sep 10 21:32:25.493359 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.493298    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda"} err="failed to get container status \"062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda\": rpc error: code = NotFound desc = could not find container \"062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda\": container with ID starting with 062363af0c19a528cf94ffb3eefb12e25afbfa9ff71d385c1c4f2f5f9f16cdda not found: ID does not exist"
Sep 10 21:32:25.504377 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.504350    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"]
Sep 10 21:32:25.508658 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.508634    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-650a5a7f-kserve-c9b875d6c-xg4g9"]
Sep 10 21:32:25.573123 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.573096    2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-dshm\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:25.573123 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.573120    2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-tls-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:25.573273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:25.573131    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrq6x\" (UniqueName: \"kubernetes.io/projected/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03-kube-api-access-vrq6x\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:26.301896 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:26.301862    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" path="/var/lib/kubelet/pods/e6ea3f5f-54a1-4687-9916-b6c42f0c3a03/volumes"
Sep 10 21:32:33.347361 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.347324    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-58fsj"]
Sep 10 21:32:33.347802 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.347698    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" containerName="main"
Sep 10 21:32:33.347802 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.347711    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" containerName="main"
Sep 10 21:32:33.347802 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.347776    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6ea3f5f-54a1-4687-9916-b6c42f0c3a03" containerName="main"
Sep 10 21:32:33.350710 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.350687    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.361932 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.361901    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-58fsj"]
Sep 10 21:32:33.445952 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.445914    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pz8f\" (UniqueName: \"kubernetes.io/projected/e6ec877e-d094-4044-b7ee-9889bddcb571-kube-api-access-9pz8f\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.446133 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.446106    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6ec877e-d094-4044-b7ee-9889bddcb571-maas-api-tls\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.446240 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.446224    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ec877e-d094-4044-b7ee-9889bddcb571-openshift-service-ca\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.446300 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.446256    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6ec877e-d094-4044-b7ee-9889bddcb571-metrics-certs\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.547438 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.547385    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6ec877e-d094-4044-b7ee-9889bddcb571-metrics-certs\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.547605 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.547451    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pz8f\" (UniqueName: \"kubernetes.io/projected/e6ec877e-d094-4044-b7ee-9889bddcb571-kube-api-access-9pz8f\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.547605 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.547485    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6ec877e-d094-4044-b7ee-9889bddcb571-maas-api-tls\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.547605 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.547539    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ec877e-d094-4044-b7ee-9889bddcb571-openshift-service-ca\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.548128 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.548099    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ec877e-d094-4044-b7ee-9889bddcb571-openshift-service-ca\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.549930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.549899    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6ec877e-d094-4044-b7ee-9889bddcb571-maas-api-tls\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.550059 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.549930    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6ec877e-d094-4044-b7ee-9889bddcb571-metrics-certs\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.555515 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.555491    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pz8f\" (UniqueName: \"kubernetes.io/projected/e6ec877e-d094-4044-b7ee-9889bddcb571-kube-api-access-9pz8f\") pod \"maas-api-559477f47-58fsj\" (UID: \"e6ec877e-d094-4044-b7ee-9889bddcb571\") " pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.663973 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.663947    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:33.943104 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:33.943078    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-58fsj"]
Sep 10 21:32:33.945505 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:32:33.945473    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ec877e_d094_4044_b7ee_9889bddcb571.slice/crio-55fc31f507436268cd42e8a54fcb55224981134fbbcaae318462261aae8f37f0 WatchSource:0}: Error finding container 55fc31f507436268cd42e8a54fcb55224981134fbbcaae318462261aae8f37f0: Status 404 returned error can't find the container with id 55fc31f507436268cd42e8a54fcb55224981134fbbcaae318462261aae8f37f0
Sep 10 21:32:34.531149 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:34.531114    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj" event={"ID":"e6ec877e-d094-4044-b7ee-9889bddcb571","Type":"ContainerStarted","Data":"ebd596a7407f992a465011775cbfa6ad8cb6175c7204e17c09871c3bf76b8f46"}
Sep 10 21:32:34.531653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:34.531172    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:34.531653 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:34.531190    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj" event={"ID":"e6ec877e-d094-4044-b7ee-9889bddcb571","Type":"ContainerStarted","Data":"55fc31f507436268cd42e8a54fcb55224981134fbbcaae318462261aae8f37f0"}
Sep 10 21:32:34.565851 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:34.565797    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj" podStartSLOduration=1.181277995 podStartE2EDuration="1.565778703s" totalImagesPullingTime="384.500708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:34.564437712 +0000 UTC m=+2048.890198624" watchObservedRunningTime="2026-09-10 21:32:34.565778703 +0000 UTC m=+2048.891539656"
Sep 10 21:32:40.541294 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:40.541258    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-559477f47-58fsj"
Sep 10 21:32:47.864169 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:47.864132    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"]
Sep 10 21:32:47.864681 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:47.864342    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn" podUID="964998e7-4388-4a6b-9e30-6a70ea7620c9" containerName="maas-api" containerID="cri-o://12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085" gracePeriod=30
Sep 10 21:32:49.114890 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.114863    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:32:49.292438 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.292324    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs\") pod \"964998e7-4388-4a6b-9e30-6a70ea7620c9\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") "
Sep 10 21:32:49.292623 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.292445    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk4n5\" (UniqueName: \"kubernetes.io/projected/964998e7-4388-4a6b-9e30-6a70ea7620c9-kube-api-access-sk4n5\") pod \"964998e7-4388-4a6b-9e30-6a70ea7620c9\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") "
Sep 10 21:32:49.292623 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.292489    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls\") pod \"964998e7-4388-4a6b-9e30-6a70ea7620c9\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") "
Sep 10 21:32:49.292623 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.292557    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964998e7-4388-4a6b-9e30-6a70ea7620c9-openshift-service-ca\") pod \"964998e7-4388-4a6b-9e30-6a70ea7620c9\" (UID: \"964998e7-4388-4a6b-9e30-6a70ea7620c9\") "
Sep 10 21:32:49.292949 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.292922    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/964998e7-4388-4a6b-9e30-6a70ea7620c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "964998e7-4388-4a6b-9e30-6a70ea7620c9" (UID: "964998e7-4388-4a6b-9e30-6a70ea7620c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:32:49.294473 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.294438    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "964998e7-4388-4a6b-9e30-6a70ea7620c9" (UID: "964998e7-4388-4a6b-9e30-6a70ea7620c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:49.294593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.294554    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "964998e7-4388-4a6b-9e30-6a70ea7620c9" (UID: "964998e7-4388-4a6b-9e30-6a70ea7620c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:49.294662 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.294622    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/964998e7-4388-4a6b-9e30-6a70ea7620c9-kube-api-access-sk4n5" (OuterVolumeSpecName: "kube-api-access-sk4n5") pod "964998e7-4388-4a6b-9e30-6a70ea7620c9" (UID: "964998e7-4388-4a6b-9e30-6a70ea7620c9"). InnerVolumeSpecName "kube-api-access-sk4n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:49.394131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.394097    2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964998e7-4388-4a6b-9e30-6a70ea7620c9-openshift-service-ca\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:49.394131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.394126    2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-metrics-certs\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:49.394131 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.394137    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk4n5\" (UniqueName: \"kubernetes.io/projected/964998e7-4388-4a6b-9e30-6a70ea7620c9-kube-api-access-sk4n5\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:49.394352 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.394147    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964998e7-4388-4a6b-9e30-6a70ea7620c9-maas-api-tls\") on node \"ip-10-0-128-196.ec2.internal\" DevicePath \"\""
Sep 10 21:32:49.605716 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.605625    2584 generic.go:356] "Generic (PLEG): container finished" podID="964998e7-4388-4a6b-9e30-6a70ea7620c9" containerID="12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085" exitCode=0
Sep 10 21:32:49.605716 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.605673    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"
Sep 10 21:32:49.605930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.605669    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn" event={"ID":"964998e7-4388-4a6b-9e30-6a70ea7620c9","Type":"ContainerDied","Data":"12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085"}
Sep 10 21:32:49.605930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.605800    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn" event={"ID":"964998e7-4388-4a6b-9e30-6a70ea7620c9","Type":"ContainerDied","Data":"16eb7666fa81d7babdd12888d39445836ae96fc93c86816f2d2619426e96a35e"}
Sep 10 21:32:49.605930 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.605826    2584 scope.go:117] "RemoveContainer" containerID="12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085"
Sep 10 21:32:49.614483 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.614464    2584 scope.go:117] "RemoveContainer" containerID="12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085"
Sep 10 21:32:49.614763 ip-10-0-128-196 kubenswrapper[2584]: E0910 21:32:49.614745    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085\": container with ID starting with 12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085 not found: ID does not exist" containerID="12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085"
Sep 10 21:32:49.614813 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.614773    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085"} err="failed to get container status \"12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085\": rpc error: code = NotFound desc = could not find container \"12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085\": container with ID starting with 12719914e83d3ad0b3cc7d9a1ce5ac2ca2d2f7637a71d25f33ffb83a446e5085 not found: ID does not exist"
Sep 10 21:32:49.628530 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.628502    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"]
Sep 10 21:32:49.632171 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:49.632150    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-hfwsn"]
Sep 10 21:32:50.301294 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:32:50.301258    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="964998e7-4388-4a6b-9e30-6a70ea7620c9" path="/var/lib/kubelet/pods/964998e7-4388-4a6b-9e30-6a70ea7620c9/volumes"
Sep 10 21:33:08.085139 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:08.085096    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gzcv5_13c47c34-0c80-4fae-9200-98fae6ef10ca/manager/0.log"
Sep 10 21:33:08.206229 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:08.206196    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5tb8w_a14f04c0-88a8-447a-bddd-c536626ea4f9/manager/0.log"
Sep 10 21:33:08.325166 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:08.325132    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76f4d89c9c-5k6m5_36817527-9b94-484a-9dae-01fa69a2d3fe/manager/0.log"
Sep 10 21:33:08.923818 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:08.923786    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-zlg2c_9c037f75-ddf6-4e20-8a0a-eb3f64b26375/manager/0.log"
Sep 10 21:33:08.931608 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:08.931582    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-zlg2c_9c037f75-ddf6-4e20-8a0a-eb3f64b26375/copy-manifests/0.log"
Sep 10 21:33:09.266069 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:09.265983    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-559477f47-58fsj_e6ec877e-d094-4044-b7ee-9889bddcb571/maas-api/0.log"
Sep 10 21:33:09.688986 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:09.688951    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-zbftp_21e1e403-ce0b-4005-9b1b-6534321d5c81/postgres/0.log"
Sep 10 21:33:10.561040 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:10.561007    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-w27hq_f02d78cf-792b-466e-bba9-9406d2257d58/manager/0.log"
Sep 10 21:33:10.678278 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:10.678247    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mkq46_5b3552ad-b1c4-4e48-a97e-862e12149dba/manager/0.log"
Sep 10 21:33:10.785328 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:10.785296    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zs2hn_6faf7168-06ee-44fc-bc37-4c7aae6e8c5b/kuadrant-console-plugin/0.log"
Sep 10 21:33:10.904547 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:10.904520    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-mhg4d_6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd/manager/0.log"
Sep 10 21:33:11.154212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:11.154182    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-8ttsw_e1479352-9c28-4bfd-92ae-412a7fb51bf8/manager/0.log"
Sep 10 21:33:11.724273 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:11.724242    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5d6sk_8dc8b3fe-548b-46f1-a775-f1f7c765ac77/kube-auth-proxy/0.log"
Sep 10 21:33:12.068514 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:12.068438    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ttz9p_f450a502-4479-474c-a9b3-c403c58f3f3b/payload-pre-processing/0.log"
Sep 10 21:33:12.309480 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:12.309446    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-596c47f5b-6k94l_314e11aa-c02f-4cad-a770-24604cebd511/router/0.log"
Sep 10 21:33:12.860108 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:12.860075    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-4m49h_5a49c2dc-ff84-4d40-9cfb-b7c54e4ef904/main/0.log"
Sep 10 21:33:13.083152 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:13.083118    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flf9zl_c9fb2ff4-5cbc-44ee-b41a-ca2d3750f6c2/main/0.log"
Sep 10 21:33:18.272953 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.272917    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jkhl9/must-gather-x6g5d"]
Sep 10 21:33:18.273341 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.273283    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="964998e7-4388-4a6b-9e30-6a70ea7620c9" containerName="maas-api"
Sep 10 21:33:18.273341 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.273294    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="964998e7-4388-4a6b-9e30-6a70ea7620c9" containerName="maas-api"
Sep 10 21:33:18.273444 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.273362    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="964998e7-4388-4a6b-9e30-6a70ea7620c9" containerName="maas-api"
Sep 10 21:33:18.276352 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.276333    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.278961 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.278935    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jkhl9\"/\"kube-root-ca.crt\""
Sep 10 21:33:18.279597 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.279577    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jkhl9\"/\"default-dockercfg-n4mbj\""
Sep 10 21:33:18.279668 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.279583    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jkhl9\"/\"openshift-service-ca.crt\""
Sep 10 21:33:18.288043 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.288021    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jkhl9/must-gather-x6g5d"]
Sep 10 21:33:18.294812 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.294785    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:18.325893 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.325866    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/55d081f2-1431-4847-b37c-0b3159745d41-must-gather-output\") pod \"must-gather-x6g5d\" (UID: \"55d081f2-1431-4847-b37c-0b3159745d41\") " pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.326053 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.325954    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p54dk\" (UniqueName: \"kubernetes.io/projected/55d081f2-1431-4847-b37c-0b3159745d41-kube-api-access-p54dk\") pod \"must-gather-x6g5d\" (UID: \"55d081f2-1431-4847-b37c-0b3159745d41\") " pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.359396 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.359363    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:18.426632 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.426597    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p54dk\" (UniqueName: \"kubernetes.io/projected/55d081f2-1431-4847-b37c-0b3159745d41-kube-api-access-p54dk\") pod \"must-gather-x6g5d\" (UID: \"55d081f2-1431-4847-b37c-0b3159745d41\") " pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.426810 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.426640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/55d081f2-1431-4847-b37c-0b3159745d41-must-gather-output\") pod \"must-gather-x6g5d\" (UID: \"55d081f2-1431-4847-b37c-0b3159745d41\") " pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.426935 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.426919    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/55d081f2-1431-4847-b37c-0b3159745d41-must-gather-output\") pod \"must-gather-x6g5d\" (UID: \"55d081f2-1431-4847-b37c-0b3159745d41\") " pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.437684 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.437657    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p54dk\" (UniqueName: \"kubernetes.io/projected/55d081f2-1431-4847-b37c-0b3159745d41-kube-api-access-p54dk\") pod \"must-gather-x6g5d\" (UID: \"55d081f2-1431-4847-b37c-0b3159745d41\") " pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.585401 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.585327    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jkhl9/must-gather-x6g5d"
Sep 10 21:33:18.854995 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.854912    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jkhl9/must-gather-x6g5d"]
Sep 10 21:33:18.858005 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:33:18.857979    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55d081f2_1431_4847_b37c_0b3159745d41.slice/crio-7700f9d8d6c6786bb64d3394bbf8583725900444ba2d80c891e6ed3f980b50ef WatchSource:0}: Error finding container 7700f9d8d6c6786bb64d3394bbf8583725900444ba2d80c891e6ed3f980b50ef: Status 404 returned error can't find the container with id 7700f9d8d6c6786bb64d3394bbf8583725900444ba2d80c891e6ed3f980b50ef
Sep 10 21:33:18.859999 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:18.859980    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:33:19.754840 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:19.754806    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jkhl9/must-gather-x6g5d" event={"ID":"55d081f2-1431-4847-b37c-0b3159745d41","Type":"ContainerStarted","Data":"7700f9d8d6c6786bb64d3394bbf8583725900444ba2d80c891e6ed3f980b50ef"}
Sep 10 21:33:20.084193 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:20.084062    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:20.763889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:20.763849    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jkhl9/must-gather-x6g5d" event={"ID":"55d081f2-1431-4847-b37c-0b3159745d41","Type":"ContainerStarted","Data":"1cdc4188a16a03fcb9905b9608b83d82dc8370467db4457612a7ea582da25d08"}
Sep 10 21:33:20.764377 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:20.763901    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jkhl9/must-gather-x6g5d" event={"ID":"55d081f2-1431-4847-b37c-0b3159745d41","Type":"ContainerStarted","Data":"d7cb14f388b62b13a0ee5848196ec71db9ab6d6f3b28d4854ac039b420a29972"}
Sep 10 21:33:20.780325 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:20.780274    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jkhl9/must-gather-x6g5d" podStartSLOduration=1.9388483079999999 podStartE2EDuration="2.780255805s" totalImagesPullingTime="841.407497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:20.778724653 +0000 UTC m=+2095.104485576" watchObservedRunningTime="2026-09-10 21:33:20.780255805 +0000 UTC m=+2095.106016725"
Sep 10 21:33:21.335021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:21.334985    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dq9qc_35386900-a40f-41d0-9c53-fe52d51e775a/global-pull-secret-syncer/0.log"
Sep 10 21:33:21.444316 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:21.444283    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b4b5r_7f4c4bf2-a6f0-4bb4-bbf7-f55373d45b40/konnectivity-agent/0.log"
Sep 10 21:33:21.560504 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:21.560471    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-196.ec2.internal_89a6a8e54f676254b2e19e6c94fb7b2e/haproxy/0.log"
Sep 10 21:33:26.320091 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:26.320003    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:33:26.328060 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:26.328026    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cs6qr_e038fb84-0426-48e5-bf8d-38200bd7d6c8/ovn-acl-logging/0.log"
Sep 10 21:33:26.605465 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:26.605357    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-w27hq_f02d78cf-792b-466e-bba9-9406d2257d58/manager/0.log"
Sep 10 21:33:26.683464 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:26.683411    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mkq46_5b3552ad-b1c4-4e48-a97e-862e12149dba/manager/0.log"
Sep 10 21:33:26.767968 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:26.767938    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zs2hn_6faf7168-06ee-44fc-bc37-4c7aae6e8c5b/kuadrant-console-plugin/0.log"
Sep 10 21:33:26.830780 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:26.830748    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-mhg4d_6c88b9f6-6d87-4c9f-a839-51a9ce8dfbcd/manager/0.log"
Sep 10 21:33:26.962040 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:26.959956    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-8ttsw_e1479352-9c28-4bfd-92ae-412a7fb51bf8/manager/0.log"
Sep 10 21:33:28.621648 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:28.621571    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-d446cbf8b-wfblv_237b84a4-64bb-452f-a55e-42023f356222/metrics-server/0.log"
Sep 10 21:33:28.926364 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:28.926331    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9kl4x_85fb2084-2834-4cdb-aa5c-b077c5c381bc/node-exporter/0.log"
Sep 10 21:33:28.950362 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:28.950338    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9kl4x_85fb2084-2834-4cdb-aa5c-b077c5c381bc/kube-rbac-proxy/0.log"
Sep 10 21:33:28.976521 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:28.976489    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9kl4x_85fb2084-2834-4cdb-aa5c-b077c5c381bc/init-textfile/0.log"
Sep 10 21:33:29.110543 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:29.110464    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a077b0b-ba04-4521-af6c-f9cc8a9936f1/prometheus/0.log"
Sep 10 21:33:29.139156 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:29.139102    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a077b0b-ba04-4521-af6c-f9cc8a9936f1/config-reloader/0.log"
Sep 10 21:33:29.162791 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:29.162760    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a077b0b-ba04-4521-af6c-f9cc8a9936f1/thanos-sidecar/0.log"
Sep 10 21:33:29.191738 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:29.191709    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a077b0b-ba04-4521-af6c-f9cc8a9936f1/kube-rbac-proxy-web/0.log"
Sep 10 21:33:29.220375 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:29.220331    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a077b0b-ba04-4521-af6c-f9cc8a9936f1/kube-rbac-proxy/0.log"
Sep 10 21:33:29.251267 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:29.250431    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a077b0b-ba04-4521-af6c-f9cc8a9936f1/kube-rbac-proxy-thanos/0.log"
Sep 10 21:33:29.278941 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:29.278913    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a077b0b-ba04-4521-af6c-f9cc8a9936f1/init-config-reloader/0.log"
Sep 10 21:33:31.379013 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.378976    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"]
Sep 10 21:33:31.384263 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.384229    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.392889 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.392853    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"]
Sep 10 21:33:31.410741 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.410594    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-lib-modules\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.410741 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.410729    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-sys\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.411008 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.410756    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-proc\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.411008 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.410785    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-podres\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.411008 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.410815    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx5ts\" (UniqueName: \"kubernetes.io/projected/91475623-355b-4623-90de-9403f70a0682-kube-api-access-xx5ts\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.511825 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.511780    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-sys\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.511836    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-proc\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.511867    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-podres\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.511894    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx5ts\" (UniqueName: \"kubernetes.io/projected/91475623-355b-4623-90de-9403f70a0682-kube-api-access-xx5ts\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.511933    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-lib-modules\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512003 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.511988    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-sys\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.511959    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-proc\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.512051    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-podres\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.512176 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.512108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91475623-355b-4623-90de-9403f70a0682-lib-modules\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.522045 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.522018    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx5ts\" (UniqueName: \"kubernetes.io/projected/91475623-355b-4623-90de-9403f70a0682-kube-api-access-xx5ts\") pod \"perf-node-gather-daemonset-zj2kd\" (UID: \"91475623-355b-4623-90de-9403f70a0682\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.701021 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.700987    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:31.981325 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:31.980016    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"]
Sep 10 21:33:31.984556 ip-10-0-128-196 kubenswrapper[2584]: W0910 21:33:31.984463    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod91475623_355b_4623_90de_9403f70a0682.slice/crio-b5bce27b685c39702519ff927a88872049afb5725527ba101d52fd00aa1c6886 WatchSource:0}: Error finding container b5bce27b685c39702519ff927a88872049afb5725527ba101d52fd00aa1c6886: Status 404 returned error can't find the container with id b5bce27b685c39702519ff927a88872049afb5725527ba101d52fd00aa1c6886
Sep 10 21:33:32.097694 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:32.097626    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7564bb99bd-g8tcb_88596fd8-abda-45f8-8457-dac6cafbf095/console/0.log"
Sep 10 21:33:32.904671 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:32.904626    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd" event={"ID":"91475623-355b-4623-90de-9403f70a0682","Type":"ContainerStarted","Data":"fc7f75e31123cc800516f964d45e298e418e038983816e5d4467f2d058a3bf29"}
Sep 10 21:33:32.905190 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:32.904697    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd" event={"ID":"91475623-355b-4623-90de-9403f70a0682","Type":"ContainerStarted","Data":"b5bce27b685c39702519ff927a88872049afb5725527ba101d52fd00aa1c6886"}
Sep 10 21:33:32.905190 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:32.904774    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:32.922189 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:32.922120    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd" podStartSLOduration=1.922099138 podStartE2EDuration="1.922099138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:32.920447498 +0000 UTC m=+2107.246208419" watchObservedRunningTime="2026-09-10 21:33:32.922099138 +0000 UTC m=+2107.247860059"
Sep 10 21:33:33.923593 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:33.923566    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8x4fw_a94dabbc-b8ff-4236-9bdd-cc3b21d92c38/dns/0.log"
Sep 10 21:33:33.952472 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:33.952440    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8x4fw_a94dabbc-b8ff-4236-9bdd-cc3b21d92c38/kube-rbac-proxy/0.log"
Sep 10 21:33:34.079898 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:34.079863    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fz9wf_09521e42-9209-4a90-bead-2f4f4b6fdc18/dns-node-resolver/0.log"
Sep 10 21:33:34.755626 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:34.755593    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sqjpl_9f7f5cd3-168b-4020-a3be-79c741f85e9b/node-ca/0.log"
Sep 10 21:33:35.964642 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:35.964609    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5d6sk_8dc8b3fe-548b-46f1-a775-f1f7c765ac77/kube-auth-proxy/0.log"
Sep 10 21:33:36.081024 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:36.080992    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ttz9p_f450a502-4479-474c-a9b3-c403c58f3f3b/payload-pre-processing/0.log"
Sep 10 21:33:36.154147 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:36.154117    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-596c47f5b-6k94l_314e11aa-c02f-4cad-a770-24604cebd511/router/0.log"
Sep 10 21:33:37.050919 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:37.050869    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q2w8b_947f3e28-930c-467e-8de6-356a58785659/serve-healthcheck-canary/0.log"
Sep 10 21:33:37.590664 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:37.590634    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-96ph4_09e38a3f-35ed-49ee-a2f3-b565197ef3b9/kube-rbac-proxy/0.log"
Sep 10 21:33:37.613154 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:37.613127    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-96ph4_09e38a3f-35ed-49ee-a2f3-b565197ef3b9/exporter/0.log"
Sep 10 21:33:37.636212 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:37.636174    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-96ph4_09e38a3f-35ed-49ee-a2f3-b565197ef3b9/extractor/0.log"
Sep 10 21:33:38.920162 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:38.920134    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-zj2kd"
Sep 10 21:33:40.031196 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:40.031162    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gzcv5_13c47c34-0c80-4fae-9200-98fae6ef10ca/manager/0.log"
Sep 10 21:33:40.358447 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:40.358345    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5tb8w_a14f04c0-88a8-447a-bddd-c536626ea4f9/manager/0.log"
Sep 10 21:33:40.433355 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:40.432516    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76f4d89c9c-5k6m5_36817527-9b94-484a-9dae-01fa69a2d3fe/manager/0.log"
Sep 10 21:33:40.793720 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:40.793685    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-zlg2c_9c037f75-ddf6-4e20-8a0a-eb3f64b26375/manager/0.log"
Sep 10 21:33:40.817292 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:40.817257    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-zlg2c_9c037f75-ddf6-4e20-8a0a-eb3f64b26375/copy-manifests/0.log"
Sep 10 21:33:42.370681 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:42.370632    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5d6sk_8dc8b3fe-548b-46f1-a775-f1f7c765ac77/kube-auth-proxy/0.log"
Sep 10 21:33:42.458414 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:42.458381    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ttz9p_f450a502-4479-474c-a9b3-c403c58f3f3b/payload-pre-processing/0.log"
Sep 10 21:33:42.516953 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:42.516912    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-596c47f5b-6k94l_314e11aa-c02f-4cad-a770-24604cebd511/router/0.log"
Sep 10 21:33:44.235052 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:44.235022    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5d6sk_8dc8b3fe-548b-46f1-a775-f1f7c765ac77/kube-auth-proxy/0.log"
Sep 10 21:33:44.328126 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:44.328089    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ttz9p_f450a502-4479-474c-a9b3-c403c58f3f3b/payload-pre-processing/0.log"
Sep 10 21:33:44.389303 ip-10-0-128-196 kubenswrapper[2584]: I0910 21:33:44.389270    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-596c47f5b-6k94l_314e11aa-c02f-4cad-a770-24604cebd511/router/0.log"