Sep 10 21:10:08.310619 ip-10-0-141-146 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 21:10:08.733177 ip-10-0-141-146 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:10:08.733177 ip-10-0-141-146 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 21:10:08.733177 ip-10-0-141-146 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:10:08.733177 ip-10-0-141-146 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 21:10:08.733177 ip-10-0-141-146 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:10:08.734973 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.734885    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 21:10:08.739550 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739533    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739552    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739556    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739560    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739563    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739566    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739569    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739572    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739575    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739577    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739580    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739582    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739586    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739589    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739591    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739594    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739597    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739600    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:08.739595 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739603    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739606    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739608    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739611    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739614    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739617    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739621    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739625    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739628    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739631    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739634    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739637    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739640    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739642    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739645    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739648    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739650    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739653    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739655    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:08.740056 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739658    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739661    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739664    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739666    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739669    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739671    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739674    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739676    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739679    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739681    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739684    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739686    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739689    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739692    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739696    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739700    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739703    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739706    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739708    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:08.740533 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739711    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739714    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739717    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739721    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739725    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739728    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739731    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739733    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739736    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739739    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739741    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739744    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739747    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739750    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739752    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739755    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739758    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739761    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739764    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739766    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:08.741000 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739769    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739771    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739774    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739776    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739779    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739781    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739783    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739786    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739789    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739792    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739796    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739799    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739802    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739804    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739807    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739809    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739812    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739815    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739817    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739820    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:08.741507 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739822    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739824    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.739827    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740551    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740559    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740561    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740564    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740567    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740585    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740589    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740591    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740594    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740597    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740600    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740602    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740606    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740610    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740613    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740616    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740619    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:08.741997 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740623    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740626    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740628    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740631    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740633    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740636    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740640    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740644    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740647    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740650    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740653    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740656    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740660    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740662    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740665    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740668    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740670    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740673    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740675    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:08.742544 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740678    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740680    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740683    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740686    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740689    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740691    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740694    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740696    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740699    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740702    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740704    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740707    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740709    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740713    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740716    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740719    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740721    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740724    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740726    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740729    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:08.743007 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740732    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740734    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740737    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740739    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740743    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740745    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740748    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740750    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740753    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740756    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740758    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740761    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740763    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740766    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740769    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740772    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740774    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740777    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740779    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740782    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:08.743524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740786    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740789    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740792    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740795    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740798    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740801    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740804    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740807    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740810    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740813    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740815    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740818    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740821    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740823    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740826    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740829    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740831    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740833    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740836    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:08.744004 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740839    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740841    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740844    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.740846    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741247    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741252    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741255    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741257    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741261    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741264    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741267    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741270    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741273    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741275    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741278    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741281    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741283    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741287    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741289    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:08.744491 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741292    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741295    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741298    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741300    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741303    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741306    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741309    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741313    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741316    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741319    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741335    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741338    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741340    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741343    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741346    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741348    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741351    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741353    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741356    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741359    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:08.744962 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741361    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741364    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741373    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741376    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741379    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741383    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741387    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741389    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741392    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741394    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741397    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741400    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741403    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741406    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741409    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741411    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741414    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741416    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741419    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741421    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:08.745463 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741424    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741427    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741429    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741431    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741434    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741437    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741439    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741442    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741444    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741447    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741449    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741452    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741454    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741457    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741461    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741467    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741469    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741472    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741476    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:08.746038 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741479    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741482    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741487    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741490    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741493    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741496    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741498    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741501    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741503    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741506    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741509    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741511    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741514    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741517    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741519    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741522    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741524    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741527    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741529    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741532    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:08.746515 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741534    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741537    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741539    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741542    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.741544    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741616    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741651    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741658    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741662    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741670    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741674    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741678    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741682    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741685    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741691    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741695    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741698    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741702    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741705    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741708    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741711    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741713    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741716    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741723    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 10 21:10:08.746996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741726    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741729    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741733    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741736    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741740    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741743    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741746    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741749    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741752    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741755    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741758    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741761    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741764    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741769    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741772    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741775    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741778    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741781    2583 flags.go:64] FLAG: --enable-server="true"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741785    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741789    2583 flags.go:64] FLAG: --event-burst="100"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741792    2583 flags.go:64] FLAG: --event-qps="50"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741795    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741800    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741803    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 10 21:10:08.747583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741807    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741810    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741813    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741815    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741819    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741822    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741825    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741828    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741831    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741834    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741836    2583 flags.go:64] FLAG: --feature-gates=""
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741840    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741843    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741846    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741849    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741852    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741856    2583 flags.go:64] FLAG: --help="false"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741858    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-146.ec2.internal"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741862    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741865    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741868    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741871    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741874    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741877    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 21:10:08.748202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741881    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741884    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741887    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741891    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741894    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741896    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741901    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741904    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741907    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741910    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741913    2583 flags.go:64] FLAG: --lock-file=""
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741916    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741918    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741921    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741927    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741930    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741933    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741935    2583 flags.go:64] FLAG: --logging-format="text"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741938    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741942    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741944    2583 flags.go:64] FLAG: --manifest-url=""
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741948    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741952    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741955    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741959    2583 flags.go:64] FLAG: --max-pods="110"
Sep 10 21:10:08.748831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741962    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741965    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741968    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741971    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741974    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741977    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741980    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741987    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741990    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741993    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741996    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.741999    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742005    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742008    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742013    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742016    2583 flags.go:64] FLAG: --port="10250"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742019    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742021    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f95b197381bfd85"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742025    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742028    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742031    2583 flags.go:64] FLAG: --register-node="true"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742034    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742037    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742042    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 10 21:10:08.749688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742045    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742047    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742054    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742057    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742060    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742063    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742066    2583 flags.go:64] FLAG: --runonce="false"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742068    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742071    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742074    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742077    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742080    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742083    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742086    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742090    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742093    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742096    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742099    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742101    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742104    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742108    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742111    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742117    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742120    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742123    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 21:10:08.750633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742127    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742130    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742133    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742136    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742139    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742142    2583 flags.go:64] FLAG: --v="2"
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742147    2583 flags.go:64] FLAG: --version="false"
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742151    2583 flags.go:64] FLAG: --vmodule=""
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742155    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.742158    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742277    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742281    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742284    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742287    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742290    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742293    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742295    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742298    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742301    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742303    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742306    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742308    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:08.751320 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742312    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742315    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742317    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742320    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742335    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742338    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742341    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742346    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742350    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742353    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742356    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742358    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742361    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742364    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742367    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742369    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742373    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742376    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742378    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742381    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:08.751887 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742384    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742387    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742389    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742392    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742394    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742397    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742400    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742402    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742405    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742407    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742410    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742412    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742415    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742417    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742420    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742422    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742425    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742427    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742430    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:08.752500 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742434    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742437    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742440    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742443    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742445    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742448    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742450    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742453    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742455    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742460    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742462    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742465    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742468    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742472    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742475    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742478    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742482    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742485    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742489    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:08.752983 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742492    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742495    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742498    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742500    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742503    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742506    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742509    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742511    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742514    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742516    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742519    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742521    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742524    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742528    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742531    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742534    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742536    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742539    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742542    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742544    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742547    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:08.753470 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742550    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742554    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742557    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742560    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742564    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742568    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742572    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.742576    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.743712    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.751862    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.751879    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.752822    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.752830    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752899    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752905    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:08.753991 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752908    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752911    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752915    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752918    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752921    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752925    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752928    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752931    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752934    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752937    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752941    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752944    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752946    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752949    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752952    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752955    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752957    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752960    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752963    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752965    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:08.754389 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752968    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752971    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752974    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752977    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752979    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752984    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752988    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752993    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752996    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.752999    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753003    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753005    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753008    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753012    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753015    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753018    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753021    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753024    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753027    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:08.754870 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753031    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753034    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753037    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753039    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753042    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753045    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753048    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753052    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753056    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753059    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753062    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753065    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753068    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753071    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753073    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753076    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753078    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753081    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753084    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:08.755346 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753086    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753089    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753092    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753095    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753098    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753100    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753103    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753106    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753108    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753111    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753113    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753116    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753118    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753121    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753124    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753126    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753129    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753131    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753134    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753137    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:08.755850 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753140    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753143    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753145    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753148    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753151    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753154    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753156    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753159    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753161    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753164    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753167    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753169    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753172    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753174    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753177    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753180    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753182    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753185    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:08.756421 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753187    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.753193    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753305    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753310    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753313    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753316    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753318    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753321    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753339    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753342    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753345    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753348    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753351    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753353    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:10:08.756860 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753357    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753361    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753365    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753368    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753371    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753374    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753376    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753380    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753384    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753387    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753390    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753393    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753395    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753398    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753401    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753404    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753406    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753409    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753412    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:10:08.757222 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753414    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753417    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753419    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753422    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753425    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753427    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753430    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753432    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753435    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753438    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753440    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753443    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753445    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753448    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753452    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753454    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753457    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753459    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753462    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753465    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:10:08.757697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753467    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753469    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753472    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753475    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753478    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753480    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753483    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753486    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753489    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753491    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753494    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753498    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753501    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753504    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753507    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753510    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753513    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753516    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753519    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753521    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:10:08.758185 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753525    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753528    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753531    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753533    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753536    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753539    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753541    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753544    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753547    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753549    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753552    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753554    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753557    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753559    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753562    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753564    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753567    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753569    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753572    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753575    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:10:08.758685 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753578    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753581    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753583    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753586    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753589    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753591    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753594    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:08.753596    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.753601    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.753728    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.756668    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.757702    2583 server.go:1019] "Starting client certificate rotation"
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.757803    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:10:08.759159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.757836    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:10:08.761202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.761187    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:10:08.762364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.762346    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:10:08.773525 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.773507    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 10 21:10:08.776645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.776629    2583 log.go:25] "Validated CRI v1 image API"
Sep 10 21:10:08.777524 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.777510    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 21:10:08.783551 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.783526    2583 fs.go:135] Filesystem UUIDs: map[4328ba82-f633-40a2-aa9d-f10a97eee5c3:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ca1213a8-9af5-4e38-9d0e-747efea7a38a:/dev/nvme0n1p4]
Sep 10 21:10:08.783626 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.783550    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 21:10:08.788977 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.788846    2583 manager.go:216] Machine: {Timestamp:2026-09-10 21:10:08.787926532 +0000 UTC m=+0.354068226 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103251 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c4c73b999216e32de649be4d16cf9 SystemUUID:ec2c4c73-b999-216e-32de-649be4d16cf9 BootID:d3d04018-cbb4-4174-a541-24990e26e828 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:04:2f:76:0c:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:04:2f:76:0c:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:cc:a8:dc:3c:8a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 21:10:08.788977 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.788964    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 21:10:08.789140 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.789080    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 21:10:08.792110 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.792084    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 21:10:08.792295 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.792113    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-146.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 21:10:08.793035 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.793022    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 21:10:08.793097 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.793055    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 21:10:08.793097 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.793067    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:10:08.793191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.793176    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:10:08.793935 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.793922    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:10:08.794692 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.794680    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 21:10:08.798250 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798237    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:10:08.798407 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798396    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 21:10:08.798466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798434    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 21:10:08.798466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798448    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 21:10:08.798561 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798468    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 21:10:08.798561 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798483    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 10 21:10:08.798561 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798497    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 21:10:08.798876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798862    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:10:08.798936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.798886    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:10:08.801039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.801023    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 21:10:08.803127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.803114    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 21:10:08.803170 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.803144    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 21:10:08.805423 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805410    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805429    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805435    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805440    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805448    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805454    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805467    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805472    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805479    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805485    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 21:10:08.805490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805493    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 21:10:08.805749 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.805499    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 21:10:08.806203 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.806195    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 21:10:08.806236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.806203    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 21:10:08.809187 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.809165    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-146.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 21:10:08.809975 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.809955    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 21:10:08.810019 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.809958    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 21:10:08.810255 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.810244    2583 server.go:1268] "Started kubelet"
Sep 10 21:10:08.810390 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.810356    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 21:10:08.810390 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.810347    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 21:10:08.810543 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.810417    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 21:10:08.811259 ip-10-0-141-146 systemd[1]: Started Kubernetes Kubelet.
Sep 10 21:10:08.812088 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.812069    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 21:10:08.812607 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.812592    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 21:10:08.817743 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.817724    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 21:10:08.818366 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.818351    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 21:10:08.819126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.819104    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 21:10:08.819126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.819128    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 21:10:08.819254 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.819167    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:08.819254 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.818116    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-146.ec2.internal.18d411e0b6cdc866  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-146.ec2.internal,UID:ip-10-0-141-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-146.ec2.internal,},FirstTimestamp:2026-09-10 21:10:08.810215526 +0000 UTC m=+0.376357219,LastTimestamp:2026-09-10 21:10:08.810215526 +0000 UTC m=+0.376357219,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-146.ec2.internal,}"
Sep 10 21:10:08.819254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.819211    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 21:10:08.819436 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.819258    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 21:10:08.819436 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.819265    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 21:10:08.819792 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.819768    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 21:10:08.820184 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820166    2583 factory.go:153] Registering CRI-O factory
Sep 10 21:10:08.820246 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820203    2583 factory.go:223] Registration of the crio container factory successfully
Sep 10 21:10:08.820284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820246    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 21:10:08.820284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820253    2583 factory.go:55] Registering systemd factory
Sep 10 21:10:08.820284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820259    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 10 21:10:08.820284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820274    2583 factory.go:103] Registering Raw factory
Sep 10 21:10:08.820440 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820288    2583 manager.go:1196] Started watching for new ooms in manager
Sep 10 21:10:08.820766 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.820745    2583 manager.go:349] Starting recovery of all containers
Sep 10 21:10:08.824256 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.824235    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bwl2"
Sep 10 21:10:08.825666 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.825643    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 21:10:08.825666 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.825651    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 21:10:08.830576 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.830523    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bwl2"
Sep 10 21:10:08.832657 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.832194    2583 manager.go:354] Recovery completed
Sep 10 21:10:08.837085 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.837069    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:08.838764 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.838734    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:08.838851 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.838776    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:08.838851 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.838792    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:08.839489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.839472    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 21:10:08.839489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.839486    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 21:10:08.839489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.839493    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:08.839669 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.839503    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:10:08.840633 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.840559    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-146.ec2.internal.18d411e0b8814075  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-146.ec2.internal,UID:ip-10-0-141-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-146.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-146.ec2.internal,},FirstTimestamp:2026-09-10 21:10:08.838754421 +0000 UTC m=+0.404896119,LastTimestamp:2026-09-10 21:10:08.838754421 +0000 UTC m=+0.404896119,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-146.ec2.internal,}"
Sep 10 21:10:08.840798 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.840782    2583 policy_none.go:49] "None policy: Start"
Sep 10 21:10:08.840857 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.840807    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 21:10:08.840857 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.840825    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 21:10:08.841945 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.841932    2583 policy_none.go:47] "Start"
Sep 10 21:10:08.842582 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.842461    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:08.842660 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.842598    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:08.842660 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.842613    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:08.868068 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.868042    2583 manager.go:337] "Starting Device Plugin manager"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.868090    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.868104    2583 server.go:87] "Starting device plugin registration server"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.868493    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.868507    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.868618    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.868701    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.868709    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.869389    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 21:10:08.885804 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.869430    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:08.913320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.913273    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 21:10:08.914602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.914585    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 21:10:08.914705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.914638    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 21:10:08.914705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.914667    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 21:10:08.914810 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.914784    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 21:10:08.917500 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.917482    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:08.968936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.968905    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:08.971129 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.971109    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:08.971233 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.971139    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:08.971233 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.971152    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:08.971233 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.971176    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-146.ec2.internal"
Sep 10 21:10:08.978340 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:08.978316    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-146.ec2.internal"
Sep 10 21:10:08.978406 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.978353    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-146.ec2.internal\": node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:08.996935 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:08.996876    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:09.015672 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.015647    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"]
Sep 10 21:10:09.015788 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.015725    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:09.017289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.017274    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:09.017391 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.017300    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:09.017391 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.017315    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:09.018647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.018635    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:09.018769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.018754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.018816 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.018785    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:09.019450 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019425    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:09.019537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019453    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:09.019537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019463    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:09.019537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019490    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:09.019537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019513    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:09.019537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019527    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:09.019819 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/419cc3454d80e28154e06a9c276d2a69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal\" (UID: \"419cc3454d80e28154e06a9c276d2a69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.019878 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019832    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/419cc3454d80e28154e06a9c276d2a69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal\" (UID: \"419cc3454d80e28154e06a9c276d2a69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.019878 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.019860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c92652107ed38ad17b29af94e128de12-config\") pod \"kube-apiserver-proxy-ip-10-0-141-146.ec2.internal\" (UID: \"c92652107ed38ad17b29af94e128de12\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.021044 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.021028    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.021089 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.021056    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:10:09.021820 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.021805    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:10:09.021886 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.021831    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:10:09.021886 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.021843    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:10:09.036900 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:09.036878    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-146.ec2.internal\" not found" node="ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.041072 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:09.041056    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-146.ec2.internal\" not found" node="ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.097390 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:09.097363    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:09.120965 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.120939    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/419cc3454d80e28154e06a9c276d2a69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal\" (UID: \"419cc3454d80e28154e06a9c276d2a69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.121028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.120969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/419cc3454d80e28154e06a9c276d2a69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal\" (UID: \"419cc3454d80e28154e06a9c276d2a69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.121028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.120985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c92652107ed38ad17b29af94e128de12-config\") pod \"kube-apiserver-proxy-ip-10-0-141-146.ec2.internal\" (UID: \"c92652107ed38ad17b29af94e128de12\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.121028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.121013    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c92652107ed38ad17b29af94e128de12-config\") pod \"kube-apiserver-proxy-ip-10-0-141-146.ec2.internal\" (UID: \"c92652107ed38ad17b29af94e128de12\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.121153 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.121021    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/419cc3454d80e28154e06a9c276d2a69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal\" (UID: \"419cc3454d80e28154e06a9c276d2a69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.121153 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.121029    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/419cc3454d80e28154e06a9c276d2a69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal\" (UID: \"419cc3454d80e28154e06a9c276d2a69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:09.198032 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:09.197985    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:09.298767 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:09.298701    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:09.339338 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:09.339293    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:10.383825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.383808    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:10.438565 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.438531    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:10.539068 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.539032    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:10.639714 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.639625    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:10.740278 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.740240    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-146.ec2.internal\" not found"
Sep 10 21:10:10.775008 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.774975    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:10.796310 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.796271    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 21:10:10.796467 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.796448    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 21:10:10.796542 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.796464    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 21:10:10.838572 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.838542    2583 apiserver.go:52] "Watching apiserver"
Sep 10 21:10:10.851990 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.851970    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 21:10:10.854142 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.854114    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-jkz74","kube-system/konnectivity-agent-g6qq8","openshift-dns/node-resolver-l7bpv","openshift-image-registry/node-ca-6knd8","openshift-multus/multus-additional-cni-plugins-5d5mz","openshift-multus/network-metrics-daemon-tz9tv","openshift-network-diagnostics/network-check-target-7b4vh","openshift-network-operator/iptables-alerter-ctbj8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs","openshift-cluster-node-tuning-operator/tuned-7vgf8","openshift-multus/multus-q9ws7","openshift-ovn-kubernetes/ovnkube-node-s8m4m"]
Sep 10 21:10:10.855789 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.855769    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.855865 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.855830    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:10.857307 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.857284    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 21:10:10.857391 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.857296    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:10.858638 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.858622    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:10.858790 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.858772    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.859239 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.859212    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 21:10:10.859328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.859240    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 21:10:10.859441 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.859424    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lh2gw\""
Sep 10 21:10:10.860155 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.860098    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.860617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.860482    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 21:10:10.860617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.860594    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 21:10:10.860617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.860604    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gtdsc\""
Sep 10 21:10:10.861783 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.861763    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.862289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.862270    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 21:10:10.862289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.862283    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ct6xd\""
Sep 10 21:10:10.862444 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.862270    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 21:10:10.862444 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.862272    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 21:10:10.863254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.863236    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:10.863342 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.863303    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:10.863342 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.863320    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 21:10:10.864346 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864315    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/493c5fe7-fae2-47d0-9f87-55f3702df7ed-tmp-dir\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.864435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864358    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5h9q\" (UniqueName: \"kubernetes.io/projected/493c5fe7-fae2-47d0-9f87-55f3702df7ed-kube-api-access-m5h9q\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.864435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864387    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/437ac578-201e-4cbf-ae6d-607f2eda40a3-host\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.864435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864413    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2tkc\" (UniqueName: \"kubernetes.io/projected/437ac578-201e-4cbf-ae6d-607f2eda40a3-kube-api-access-q2tkc\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.864595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864442    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bfd4681e-1a79-459f-b21c-118f4c0d2646-dbus\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.864595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.864595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864496    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/656d5e3d-d1a5-4045-b975-fc3390b46564-agent-certs\") pod \"konnectivity-agent-g6qq8\" (UID: \"656d5e3d-d1a5-4045-b975-fc3390b46564\") " pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:10.864595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/656d5e3d-d1a5-4045-b975-fc3390b46564-konnectivity-ca\") pod \"konnectivity-agent-g6qq8\" (UID: \"656d5e3d-d1a5-4045-b975-fc3390b46564\") " pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:10.864595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864562    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/493c5fe7-fae2-47d0-9f87-55f3702df7ed-hosts-file\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.864595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/437ac578-201e-4cbf-ae6d-607f2eda40a3-serviceca\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.864894 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.864615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bfd4681e-1a79-459f-b21c-118f4c0d2646-kubelet-config\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.865036 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.865013    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nx2q5\""
Sep 10 21:10:10.865262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.865240    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 21:10:10.865579 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.865560    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 21:10:10.865778 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.865743    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 21:10:10.865778 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.865772    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 21:10:10.866012 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.865997    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:10.866081 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.866061    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:10.867640 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.867619    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:10.869173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.869159    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.869460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.869340    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:10:10.869460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.869365    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:10:10.869616 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.869472    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 21:10:10.869616 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.869528    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vfdnx\""
Sep 10 21:10:10.869684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.869662    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 21:10:10.870315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.870300    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:10:10.870381 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.870363    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"
Sep 10 21:10:10.871037 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.871021    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.873066 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.872926    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 21:10:10.873608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.873591    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-59p66\""
Sep 10 21:10:10.873972 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.873789    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 21:10:10.874071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.874040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.874602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.874545    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 21:10:10.876217 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.876200    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal"]
Sep 10 21:10:10.876305 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.876291    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.876577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.876429    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 21:05:09 +0000 UTC" deadline="2028-04-02 17:42:42.932204682 +0000 UTC"
Sep 10 21:10:10.876577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.876473    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13676h32m32.055735402s"
Sep 10 21:10:10.877223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.877209    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dwwnr\""
Sep 10 21:10:10.877333 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.877317    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 21:10:10.877405 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.877333    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:10:10.877949 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.877936    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 21:10:10.880883 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.880864    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 21:10:10.881083 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.881065    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 21:10:10.881160 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.881085    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 21:10:10.881223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.881180    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 21:10:10.881223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.881204    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-khssb\""
Sep 10 21:10:10.881322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.881244    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 21:10:10.881594 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.881580    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 21:10:10.882799 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.882784    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwnjc\""
Sep 10 21:10:10.900176 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.900159    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:10:10.900282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.900256    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal"]
Sep 10 21:10:10.913966 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.913944    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nk7ws"
Sep 10 21:10:10.926540 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.926517    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nk7ws"
Sep 10 21:10:10.959210 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.959187    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 21:10:10.965173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965155    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-ovnkube-script-lib\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.965268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965180    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysctl-conf\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.965268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965197    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-tuned\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.965268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965224    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/437ac578-201e-4cbf-ae6d-607f2eda40a3-host\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.965268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-systemd\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.965400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965312    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-cni-binary-copy\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.965400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965319    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/437ac578-201e-4cbf-ae6d-607f2eda40a3-host\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.965400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965347    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-k8s-cni-cncf-io\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.965400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/243623e5-602c-486d-8231-4e52706f65f9-iptables-alerter-script\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:10.965400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-etc-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-var-lib-kubelet\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965424    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-system-cni-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2tkc\" (UniqueName: \"kubernetes.io/projected/437ac578-201e-4cbf-ae6d-607f2eda40a3-kube-api-access-q2tkc\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965455    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-env-overrides\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965475    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965524    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-kubelet\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-run\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-netns\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.965602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965593    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-cni-multus\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965612    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-cnibin\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965690    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5v6r\" (UniqueName: \"kubernetes.io/projected/14806076-2615-4738-8dec-31341e6df82d-kube-api-access-c5v6r\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965716    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysconfig\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d004620-97c0-43a0-a20b-8b3f3478b78c-tmp\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/656d5e3d-d1a5-4045-b975-fc3390b46564-agent-certs\") pod \"konnectivity-agent-g6qq8\" (UID: \"656d5e3d-d1a5-4045-b975-fc3390b46564\") " pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965812    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/656d5e3d-d1a5-4045-b975-fc3390b46564-konnectivity-ca\") pod \"konnectivity-agent-g6qq8\" (UID: \"656d5e3d-d1a5-4045-b975-fc3390b46564\") " pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/493c5fe7-fae2-47d0-9f87-55f3702df7ed-hosts-file\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965878    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-etc-selinux\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-var-lib-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p668m\" (UniqueName: \"kubernetes.io/projected/b3857789-0900-400c-b571-2a3980a3c7ee-kube-api-access-p668m\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965953    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/493c5fe7-fae2-47d0-9f87-55f3702df7ed-hosts-file\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.966003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.965985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-socket-dir-parent\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966006    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-system-cni-dir\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-systemd-units\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966052    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-log-socket\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-ovnkube-config\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966106    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-lib-modules\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966150    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-cni-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966153    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-kubelet\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966216    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-registration-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966257    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-device-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966302    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-cni-netd\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966346    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b563a9e-843d-4611-b193-7940b3016bcb-ovn-node-metrics-cert\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.966473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966363    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/656d5e3d-d1a5-4045-b975-fc3390b46564-konnectivity-ca\") pod \"konnectivity-agent-g6qq8\" (UID: \"656d5e3d-d1a5-4045-b975-fc3390b46564\") " pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966371    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-multus-certs\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966388    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-slash\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966405    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966419    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-sys\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966436    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9zqf\" (UniqueName: \"kubernetes.io/projected/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-kube-api-access-t9zqf\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-socket-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/243623e5-602c-486d-8231-4e52706f65f9-host-slash\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-sys-fs\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966616    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-systemd\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-modprobe-d\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pm5r\" (UniqueName: \"kubernetes.io/projected/6d004620-97c0-43a0-a20b-8b3f3478b78c-kube-api-access-8pm5r\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/493c5fe7-fae2-47d0-9f87-55f3702df7ed-tmp-dir\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.966677    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:10.967133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-ovn\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:10.966728    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret podName:bfd4681e-1a79-459f-b21c-118f4c0d2646 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:11.46671152 +0000 UTC m=+1.993675845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret") pod "global-pull-secret-syncer-jkz74" (UID: "bfd4681e-1a79-459f-b21c-118f4c0d2646") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966803    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4tdm\" (UniqueName: \"kubernetes.io/projected/7b563a9e-843d-4611-b193-7940b3016bcb-kube-api-access-r4tdm\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-cni-bin\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-hostroot\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966878    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-daemon-config\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966898    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knt5t\" (UniqueName: \"kubernetes.io/projected/243623e5-602c-486d-8231-4e52706f65f9-kube-api-access-knt5t\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966918    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966937    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysctl-d\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-cnibin\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.966981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bfd4681e-1a79-459f-b21c-118f4c0d2646-kubelet-config\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967000    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/493c5fe7-fae2-47d0-9f87-55f3702df7ed-tmp-dir\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2jlv\" (UniqueName: \"kubernetes.io/projected/577291c3-b7fc-434f-b1a7-ceff5ec93931-kube-api-access-m2jlv\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967030    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-run-netns\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967052    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bfd4681e-1a79-459f-b21c-118f4c0d2646-kubelet-config\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967054    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-kubernetes\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.967747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967082    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-host\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-conf-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-etc-kubernetes\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967164    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-node-log\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967193    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5h9q\" (UniqueName: \"kubernetes.io/projected/493c5fe7-fae2-47d0-9f87-55f3702df7ed-kube-api-access-m5h9q\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-cni-bin\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bfd4681e-1a79-459f-b21c-118f4c0d2646-dbus\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967265    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/437ac578-201e-4cbf-ae6d-607f2eda40a3-serviceca\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-cni-binary-copy\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967315    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967339    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bfd4681e-1a79-459f-b21c-118f4c0d2646-dbus\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:10.968209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.967607    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/437ac578-201e-4cbf-ae6d-607f2eda40a3-serviceca\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.969755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.969734    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/656d5e3d-d1a5-4045-b975-fc3390b46564-agent-certs\") pod \"konnectivity-agent-g6qq8\" (UID: \"656d5e3d-d1a5-4045-b975-fc3390b46564\") " pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:10.976770 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.976747    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2tkc\" (UniqueName: \"kubernetes.io/projected/437ac578-201e-4cbf-ae6d-607f2eda40a3-kube-api-access-q2tkc\") pod \"node-ca-6knd8\" (UID: \"437ac578-201e-4cbf-ae6d-607f2eda40a3\") " pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:10.977301 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:10.977286    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5h9q\" (UniqueName: \"kubernetes.io/projected/493c5fe7-fae2-47d0-9f87-55f3702df7ed-kube-api-access-m5h9q\") pod \"node-resolver-l7bpv\" (UID: \"493c5fe7-fae2-47d0-9f87-55f3702df7ed\") " pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:11.003461 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.003352    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92652107ed38ad17b29af94e128de12.slice/crio-87f64d29118f7ee4f7d664e089340dbeea6222d683ed505053e9d935f85a359f WatchSource:0}: Error finding container 87f64d29118f7ee4f7d664e089340dbeea6222d683ed505053e9d935f85a359f: Status 404 returned error can't find the container with id 87f64d29118f7ee4f7d664e089340dbeea6222d683ed505053e9d935f85a359f
Sep 10 21:10:11.003681 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.003658    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419cc3454d80e28154e06a9c276d2a69.slice/crio-4053abf672de0faaf2879f5cd813b556308030d21b71047b080a0b50255652d8 WatchSource:0}: Error finding container 4053abf672de0faaf2879f5cd813b556308030d21b71047b080a0b50255652d8: Status 404 returned error can't find the container with id 4053abf672de0faaf2879f5cd813b556308030d21b71047b080a0b50255652d8
Sep 10 21:10:11.009019 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.008998    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:10:11.067590 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067554    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-etc-selinux\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.067590 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-var-lib-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p668m\" (UniqueName: \"kubernetes.io/projected/b3857789-0900-400c-b571-2a3980a3c7ee-kube-api-access-p668m\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067620    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-socket-dir-parent\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-system-cni-dir\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-systemd-units\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067671    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-var-lib-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067706    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-system-cni-dir\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067723    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-etc-selinux\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-socket-dir-parent\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067781    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-log-socket\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067781    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-systemd-units\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-ovnkube-config\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-lib-modules\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067834    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-log-socket\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067852    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-cni-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.067866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-kubelet\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067885    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-registration-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067923    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-kubelet\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067935    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-cni-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-device-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067973    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-device-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067988    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-registration-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-cni-netd\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.067990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-lib-modules\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068030    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068010    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068058    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b563a9e-843d-4611-b193-7940b3016bcb-ovn-node-metrics-cert\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068061    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-cni-netd\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-multus-certs\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068100    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-slash\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.068587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068096    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-sys\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068147    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-multus-certs\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068157    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9zqf\" (UniqueName: \"kubernetes.io/projected/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-kube-api-access-t9zqf\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-slash\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068197    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-sys\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068181    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-socket-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.068243    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.068315    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:10:11.568281352 +0000 UTC m=+2.095245680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/243623e5-602c-486d-8231-4e52706f65f9-host-slash\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068401    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-socket-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-sys-fs\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068420    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-ovnkube-config\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068448    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-systemd\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068456    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-sys-fs\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.069363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068474    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-modprobe-d\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068496    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/243623e5-602c-486d-8231-4e52706f65f9-host-slash\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068499    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pm5r\" (UniqueName: \"kubernetes.io/projected/6d004620-97c0-43a0-a20b-8b3f3478b78c-kube-api-access-8pm5r\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068534    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-ovn\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068537    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-systemd\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4tdm\" (UniqueName: \"kubernetes.io/projected/7b563a9e-843d-4611-b193-7940b3016bcb-kube-api-access-r4tdm\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-cni-bin\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-hostroot\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068621    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-modprobe-d\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-run-ovn\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068654    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-cni-bin\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068678    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-hostroot\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-daemon-config\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knt5t\" (UniqueName: \"kubernetes.io/projected/243623e5-602c-486d-8231-4e52706f65f9-kube-api-access-knt5t\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068753    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.070243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysctl-d\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068879    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-cnibin\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068884    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysctl-d\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-cnibin\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068919    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2jlv\" (UniqueName: \"kubernetes.io/projected/577291c3-b7fc-434f-b1a7-ceff5ec93931-kube-api-access-m2jlv\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068937    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-run-netns\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-kubernetes\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-host\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.068987    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-conf-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-run-netns\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-etc-kubernetes\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069032    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-node-log\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069044    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-conf-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069060    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-cni-bin\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069083    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-node-log\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069086    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-cni-binary-copy\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069102    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-kubernetes\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.071134 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069125    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-multus-daemon-config\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069139    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-ovnkube-script-lib\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069086    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-etc-kubernetes\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069154    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-cni-bin\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069175    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysctl-conf\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-tuned\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-systemd\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-cni-binary-copy\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069287    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-k8s-cni-cncf-io\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069311    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/243623e5-602c-486d-8231-4e52706f65f9-iptables-alerter-script\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-etc-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysctl-conf\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069356    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-var-lib-kubelet\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-k8s-cni-cncf-io\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-system-cni-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-env-overrides\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069444    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-system-cni-dir\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069459    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-kubelet\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-run\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069551    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-netns\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-cni-multus\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069585    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/577291c3-b7fc-434f-b1a7-ceff5ec93931-cni-binary-copy\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-cnibin\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5v6r\" (UniqueName: \"kubernetes.io/projected/14806076-2615-4738-8dec-31341e6df82d-kube-api-access-c5v6r\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysconfig\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069699    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d004620-97c0-43a0-a20b-8b3f3478b78c-tmp\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069702    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-ovnkube-script-lib\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069772    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-run\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-var-lib-kubelet\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069712    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-systemd\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.069805    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-cni-binary-copy\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070009    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-host\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070053    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-etc-openvswitch\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.072588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070100    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-cnibin\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070107    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-run-netns\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070125    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7b563a9e-843d-4611-b193-7940b3016bcb-env-overrides\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-host-var-lib-cni-multus\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-sysconfig\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070184    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b563a9e-843d-4611-b193-7940b3016bcb-host-kubelet\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070190    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14806076-2615-4738-8dec-31341e6df82d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/577291c3-b7fc-434f-b1a7-ceff5ec93931-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/243623e5-602c-486d-8231-4e52706f65f9-iptables-alerter-script\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.070948    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7b563a9e-843d-4611-b193-7940b3016bcb-ovn-node-metrics-cert\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.071701    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6d004620-97c0-43a0-a20b-8b3f3478b78c-etc-tuned\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.073075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.072215    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d004620-97c0-43a0-a20b-8b3f3478b78c-tmp\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.076073 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.076051    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:11.076187 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.076080    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:11.076187 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.076095    2583 projected.go:196] Error preparing data for projected volume kube-api-access-h5hrx for pod openshift-network-diagnostics/network-check-target-7b4vh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:11.076187 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.076112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p668m\" (UniqueName: \"kubernetes.io/projected/b3857789-0900-400c-b571-2a3980a3c7ee-kube-api-access-p668m\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:11.076187 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.076152    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx podName:f48e8a2c-2f17-4fae-ae1c-2ec72801feb2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:11.576135679 +0000 UTC m=+2.103100026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h5hrx" (UniqueName: "kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx") pod "network-check-target-7b4vh" (UID: "f48e8a2c-2f17-4fae-ae1c-2ec72801feb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:11.077694 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.077658    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9zqf\" (UniqueName: \"kubernetes.io/projected/95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5-kube-api-access-t9zqf\") pod \"multus-q9ws7\" (UID: \"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5\") " pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.077888 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.077867    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pm5r\" (UniqueName: \"kubernetes.io/projected/6d004620-97c0-43a0-a20b-8b3f3478b78c-kube-api-access-8pm5r\") pod \"tuned-7vgf8\" (UID: \"6d004620-97c0-43a0-a20b-8b3f3478b78c\") " pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.078016 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.078001    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4tdm\" (UniqueName: \"kubernetes.io/projected/7b563a9e-843d-4611-b193-7940b3016bcb-kube-api-access-r4tdm\") pod \"ovnkube-node-s8m4m\" (UID: \"7b563a9e-843d-4611-b193-7940b3016bcb\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.078118 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.078102    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knt5t\" (UniqueName: \"kubernetes.io/projected/243623e5-602c-486d-8231-4e52706f65f9-kube-api-access-knt5t\") pod \"iptables-alerter-ctbj8\" (UID: \"243623e5-602c-486d-8231-4e52706f65f9\") " pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:11.078495 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.078481    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2jlv\" (UniqueName: \"kubernetes.io/projected/577291c3-b7fc-434f-b1a7-ceff5ec93931-kube-api-access-m2jlv\") pod \"multus-additional-cni-plugins-5d5mz\" (UID: \"577291c3-b7fc-434f-b1a7-ceff5ec93931\") " pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.078495 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.078490    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5v6r\" (UniqueName: \"kubernetes.io/projected/14806076-2615-4738-8dec-31341e6df82d-kube-api-access-c5v6r\") pod \"aws-ebs-csi-driver-node-scpbs\" (UID: \"14806076-2615-4738-8dec-31341e6df82d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.186962 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.186887    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:11.196891 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.196865    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod656d5e3d_d1a5_4045_b975_fc3390b46564.slice/crio-ec23ad8b28163ea5e3303a0d00dc17946b1a8b63701568c34c11adcaafa6e58e WatchSource:0}: Error finding container ec23ad8b28163ea5e3303a0d00dc17946b1a8b63701568c34c11adcaafa6e58e: Status 404 returned error can't find the container with id ec23ad8b28163ea5e3303a0d00dc17946b1a8b63701568c34c11adcaafa6e58e
Sep 10 21:10:11.211684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.211667    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l7bpv"
Sep 10 21:10:11.219697 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.219674    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493c5fe7_fae2_47d0_9f87_55f3702df7ed.slice/crio-a5aea6524701afbf365ee470a5624d887bb19186ab5aa959347e156db81093e2 WatchSource:0}: Error finding container a5aea6524701afbf365ee470a5624d887bb19186ab5aa959347e156db81093e2: Status 404 returned error can't find the container with id a5aea6524701afbf365ee470a5624d887bb19186ab5aa959347e156db81093e2
Sep 10 21:10:11.225649 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.225635    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6knd8"
Sep 10 21:10:11.233814 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.233790    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437ac578_201e_4cbf_ae6d_607f2eda40a3.slice/crio-b901afa91ea452095f7e6318d7b466058b5fefbd884a9f24630d98a441fc7bbb WatchSource:0}: Error finding container b901afa91ea452095f7e6318d7b466058b5fefbd884a9f24630d98a441fc7bbb: Status 404 returned error can't find the container with id b901afa91ea452095f7e6318d7b466058b5fefbd884a9f24630d98a441fc7bbb
Sep 10 21:10:11.240897 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.240880    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5d5mz"
Sep 10 21:10:11.246628 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.246612    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ctbj8"
Sep 10 21:10:11.248524 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.248492    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577291c3_b7fc_434f_b1a7_ceff5ec93931.slice/crio-90823b30daa538bfc521bb6ccb67e9995d02b511151b6b5d83b014ee7746b625 WatchSource:0}: Error finding container 90823b30daa538bfc521bb6ccb67e9995d02b511151b6b5d83b014ee7746b625: Status 404 returned error can't find the container with id 90823b30daa538bfc521bb6ccb67e9995d02b511151b6b5d83b014ee7746b625
Sep 10 21:10:11.255362 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.255339    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243623e5_602c_486d_8231_4e52706f65f9.slice/crio-fc11a705dd7ffc93c2edb7806ac10d960562efa9ec7272838a52b11b7edca368 WatchSource:0}: Error finding container fc11a705dd7ffc93c2edb7806ac10d960562efa9ec7272838a52b11b7edca368: Status 404 returned error can't find the container with id fc11a705dd7ffc93c2edb7806ac10d960562efa9ec7272838a52b11b7edca368
Sep 10 21:10:11.276848 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.276823    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs"
Sep 10 21:10:11.284688 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.284666    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14806076_2615_4738_8dec_31341e6df82d.slice/crio-836eb785f58b8bea0e6c0e5ebcaef04a6fc9d830b116fa185d102db0c4b53473 WatchSource:0}: Error finding container 836eb785f58b8bea0e6c0e5ebcaef04a6fc9d830b116fa185d102db0c4b53473: Status 404 returned error can't find the container with id 836eb785f58b8bea0e6c0e5ebcaef04a6fc9d830b116fa185d102db0c4b53473
Sep 10 21:10:11.287201 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.287179    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7vgf8"
Sep 10 21:10:11.294575 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.294557    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d004620_97c0_43a0_a20b_8b3f3478b78c.slice/crio-8d7ae6c9c2b609e1fd57afeeb2a0b37fff0cceb9e2c97c187ea10092c6e9a279 WatchSource:0}: Error finding container 8d7ae6c9c2b609e1fd57afeeb2a0b37fff0cceb9e2c97c187ea10092c6e9a279: Status 404 returned error can't find the container with id 8d7ae6c9c2b609e1fd57afeeb2a0b37fff0cceb9e2c97c187ea10092c6e9a279
Sep 10 21:10:11.309746 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.309725    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q9ws7"
Sep 10 21:10:11.313456 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.313385    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:11.319062 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.319042    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ae0a83_f650_4cb9_8e49_3f9ba32f7ce5.slice/crio-5cf9a858a5d4521d86c6a702526e78df634b65070cb242a66ffead2833bcaa75 WatchSource:0}: Error finding container 5cf9a858a5d4521d86c6a702526e78df634b65070cb242a66ffead2833bcaa75: Status 404 returned error can't find the container with id 5cf9a858a5d4521d86c6a702526e78df634b65070cb242a66ffead2833bcaa75
Sep 10 21:10:11.322271 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:11.322251    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b563a9e_843d_4611_b193_7940b3016bcb.slice/crio-a92fd8772677cb2dcf7b5d16b82aa3b6d95058bdfe93bfc0bd1bf1dc43860c75 WatchSource:0}: Error finding container a92fd8772677cb2dcf7b5d16b82aa3b6d95058bdfe93bfc0bd1bf1dc43860c75: Status 404 returned error can't find the container with id a92fd8772677cb2dcf7b5d16b82aa3b6d95058bdfe93bfc0bd1bf1dc43860c75
Sep 10 21:10:11.330685 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.330667    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:11.427217 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.425650    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:11.473563 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.473439    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:11.473716 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.473619    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:11.473716 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.473693    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret podName:bfd4681e-1a79-459f-b21c-118f4c0d2646 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:12.473673945 +0000 UTC m=+3.000638287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret") pod "global-pull-secret-syncer-jkz74" (UID: "bfd4681e-1a79-459f-b21c-118f4c0d2646") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:11.573892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.573861    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:11.574032 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.573983    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:11.574089 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.574036    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:10:12.574022339 +0000 UTC m=+3.100986665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:11.674326 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.674245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:11.674551 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.674530    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:11.674636 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.674561    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:11.674636 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.674576    2583 projected.go:196] Error preparing data for projected volume kube-api-access-h5hrx for pod openshift-network-diagnostics/network-check-target-7b4vh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:11.674636 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.674636    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx podName:f48e8a2c-2f17-4fae-ae1c-2ec72801feb2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:12.674617657 +0000 UTC m=+3.201581982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5hrx" (UniqueName: "kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx") pod "network-check-target-7b4vh" (UID: "f48e8a2c-2f17-4fae-ae1c-2ec72801feb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:11.929580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.929237    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:05:10 +0000 UTC" deadline="2028-04-16 13:11:51.831510322 +0000 UTC"
Sep 10 21:10:11.929580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.929536    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14008h1m39.901979988s"
Sep 10 21:10:11.949418 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.949217    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:11.957712 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.956750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:11.957712 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.956871    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:11.957712 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.957240    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:11.957712 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:11.957350    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:11.983338 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:11.979901    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q9ws7" event={"ID":"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5","Type":"ContainerStarted","Data":"5cf9a858a5d4521d86c6a702526e78df634b65070cb242a66ffead2833bcaa75"}
Sep 10 21:10:12.004152 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.004026    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7vgf8" event={"ID":"6d004620-97c0-43a0-a20b-8b3f3478b78c","Type":"ContainerStarted","Data":"8d7ae6c9c2b609e1fd57afeeb2a0b37fff0cceb9e2c97c187ea10092c6e9a279"}
Sep 10 21:10:12.013332 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.013235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ctbj8" event={"ID":"243623e5-602c-486d-8231-4e52706f65f9","Type":"ContainerStarted","Data":"fc11a705dd7ffc93c2edb7806ac10d960562efa9ec7272838a52b11b7edca368"}
Sep 10 21:10:12.024859 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.024739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l7bpv" event={"ID":"493c5fe7-fae2-47d0-9f87-55f3702df7ed","Type":"ContainerStarted","Data":"a5aea6524701afbf365ee470a5624d887bb19186ab5aa959347e156db81093e2"}
Sep 10 21:10:12.038099 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.036890    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g6qq8" event={"ID":"656d5e3d-d1a5-4045-b975-fc3390b46564","Type":"ContainerStarted","Data":"ec23ad8b28163ea5e3303a0d00dc17946b1a8b63701568c34c11adcaafa6e58e"}
Sep 10 21:10:12.042729 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.042606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"a92fd8772677cb2dcf7b5d16b82aa3b6d95058bdfe93bfc0bd1bf1dc43860c75"}
Sep 10 21:10:12.054952 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.054919    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs" event={"ID":"14806076-2615-4738-8dec-31341e6df82d","Type":"ContainerStarted","Data":"836eb785f58b8bea0e6c0e5ebcaef04a6fc9d830b116fa185d102db0c4b53473"}
Sep 10 21:10:12.058001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.057947    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerStarted","Data":"90823b30daa538bfc521bb6ccb67e9995d02b511151b6b5d83b014ee7746b625"}
Sep 10 21:10:12.060987 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.060964    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6knd8" event={"ID":"437ac578-201e-4cbf-ae6d-607f2eda40a3","Type":"ContainerStarted","Data":"b901afa91ea452095f7e6318d7b466058b5fefbd884a9f24630d98a441fc7bbb"}
Sep 10 21:10:12.071675 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.071650    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal" event={"ID":"c92652107ed38ad17b29af94e128de12","Type":"ContainerStarted","Data":"87f64d29118f7ee4f7d664e089340dbeea6222d683ed505053e9d935f85a359f"}
Sep 10 21:10:12.093894 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.093863    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal" event={"ID":"419cc3454d80e28154e06a9c276d2a69","Type":"ContainerStarted","Data":"4053abf672de0faaf2879f5cd813b556308030d21b71047b080a0b50255652d8"}
Sep 10 21:10:12.481202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.481038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:12.481202 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.481185    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:12.481444 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.481254    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret podName:bfd4681e-1a79-459f-b21c-118f4c0d2646 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:14.481235646 +0000 UTC m=+5.008199986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret") pod "global-pull-secret-syncer-jkz74" (UID: "bfd4681e-1a79-459f-b21c-118f4c0d2646") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:12.582404 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.582362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:12.582613 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.582571    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:12.582685 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.582642    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:10:14.582622177 +0000 UTC m=+5.109586525 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:12.683605 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.683561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:12.683771 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.683736    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:12.683771 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.683758    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:12.683771 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.683770    2583 projected.go:196] Error preparing data for projected volume kube-api-access-h5hrx for pod openshift-network-diagnostics/network-check-target-7b4vh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:12.683935 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.683826    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx podName:f48e8a2c-2f17-4fae-ae1c-2ec72801feb2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:14.683808675 +0000 UTC m=+5.210773006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5hrx" (UniqueName: "kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx") pod "network-check-target-7b4vh" (UID: "f48e8a2c-2f17-4fae-ae1c-2ec72801feb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:12.930226 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.930103    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:05:10 +0000 UTC" deadline="2028-03-10 11:33:25.65407441 +0000 UTC"
Sep 10 21:10:12.930226 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.930143    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13118h23m12.723935547s"
Sep 10 21:10:12.955415 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:12.954894    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:12.955415 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:12.955021    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:13.637605 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:13.637324    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:10:13.958829 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:13.958743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:13.959259 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:13.958888    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:13.959316 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:13.959266    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:13.959452 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:13.959364    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:14.499938 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:14.499899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:14.500145 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.500087    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:14.500209 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.500169    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret podName:bfd4681e-1a79-459f-b21c-118f4c0d2646 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:18.500148468 +0000 UTC m=+9.027112814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret") pod "global-pull-secret-syncer-jkz74" (UID: "bfd4681e-1a79-459f-b21c-118f4c0d2646") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:14.601499 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:14.601462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:14.601673 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.601624    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:14.601747 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.601703    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:10:18.601683112 +0000 UTC m=+9.128647455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:14.702020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:14.701983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:14.702197 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.702146    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:14.702197 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.702166    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:14.702197 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.702182    2583 projected.go:196] Error preparing data for projected volume kube-api-access-h5hrx for pod openshift-network-diagnostics/network-check-target-7b4vh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:14.702362 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.702249    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx podName:f48e8a2c-2f17-4fae-ae1c-2ec72801feb2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:18.702228354 +0000 UTC m=+9.229192693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5hrx" (UniqueName: "kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx") pod "network-check-target-7b4vh" (UID: "f48e8a2c-2f17-4fae-ae1c-2ec72801feb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:14.954567 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:14.954444    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:14.954739 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:14.954590    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:15.958059 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:15.958033    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:15.958553 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:15.958132    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:15.958553 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:15.958381    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:15.958553 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:15.958437    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:16.954708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:16.954671    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:16.954914 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:16.954808    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:17.954657 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:17.954620    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:17.955114 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:17.954749    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:17.955114 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:17.955107    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:17.955226 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:17.955203    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:18.536782 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:18.536741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:18.536960 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.536885    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:18.536960 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.536952    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret podName:bfd4681e-1a79-459f-b21c-118f4c0d2646 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:26.536935691 +0000 UTC m=+17.063900016 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret") pod "global-pull-secret-syncer-jkz74" (UID: "bfd4681e-1a79-459f-b21c-118f4c0d2646") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:18.637633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:18.637540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:18.637803 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.637711    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:18.637803 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.637797    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:10:26.637774366 +0000 UTC m=+17.164738697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:18.738348 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:18.738250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:18.738523 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.738462    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:18.738523 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.738483    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:18.738643 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.738496    2583 projected.go:196] Error preparing data for projected volume kube-api-access-h5hrx for pod openshift-network-diagnostics/network-check-target-7b4vh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:18.738643 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.738631    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx podName:f48e8a2c-2f17-4fae-ae1c-2ec72801feb2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:26.738604322 +0000 UTC m=+17.265568663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5hrx" (UniqueName: "kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx") pod "network-check-target-7b4vh" (UID: "f48e8a2c-2f17-4fae-ae1c-2ec72801feb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:18.954772 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:18.954738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:18.955225 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:18.954867    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:19.955663 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:19.955620    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:19.956146 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:19.955752    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:19.956146 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:19.956075    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:19.956241 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:19.956166    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:20.955080 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:20.955043    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:20.955268 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:20.955178    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:21.954804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:21.954765    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:21.955287 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:21.954907    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:21.955287 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:21.954765    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:21.955394 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:21.955303    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:22.954234 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:22.954158    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:22.954391 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:22.954269    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:23.954775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:23.954732    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:23.955224 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:23.954746    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:23.955224 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:23.954878    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:23.955224 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:23.954929    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:24.954339 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:24.954302    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:24.954553 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:24.954437    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:25.955063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:25.955022    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:25.955490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:25.955027    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:25.955490 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:25.955147    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:25.955490 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:25.955284    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:26.599111 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:26.599024    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:26.599403 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.599160    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:26.599403 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.599222    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret podName:bfd4681e-1a79-459f-b21c-118f4c0d2646 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:42.599205884 +0000 UTC m=+33.126170225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret") pod "global-pull-secret-syncer-jkz74" (UID: "bfd4681e-1a79-459f-b21c-118f4c0d2646") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:26.699915 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:26.699866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:26.700074 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.700015    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:26.700141 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.700096    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:10:42.700073149 +0000 UTC m=+33.227037490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:26.801028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:26.800996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:26.801198 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.801179    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:26.801260 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.801202    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:26.801260 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.801212    2583 projected.go:196] Error preparing data for projected volume kube-api-access-h5hrx for pod openshift-network-diagnostics/network-check-target-7b4vh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:26.801324 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.801264    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx podName:f48e8a2c-2f17-4fae-ae1c-2ec72801feb2 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:42.801245118 +0000 UTC m=+33.328209444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5hrx" (UniqueName: "kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx") pod "network-check-target-7b4vh" (UID: "f48e8a2c-2f17-4fae-ae1c-2ec72801feb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:26.954511 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:26.954413    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:26.954651 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:26.954556    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:27.954767 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:27.954731    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:27.955233 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:27.954732    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:27.955233 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:27.954860    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:27.955233 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:27.954958    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:28.954668 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:28.954634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:28.954820 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:28.954757    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:29.956821 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:29.956801    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:29.957251 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:29.956885    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:29.957251 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:29.956936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:29.957251 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:29.957015    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:30.155280 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:30.155036    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal" event={"ID":"c92652107ed38ad17b29af94e128de12","Type":"ContainerStarted","Data":"a62be73cb02780351f95de24e86d3e04b5ff09af17a6b82e396c7d5f1274685e"}
Sep 10 21:10:30.160631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:30.160604    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7vgf8" event={"ID":"6d004620-97c0-43a0-a20b-8b3f3478b78c","Type":"ContainerStarted","Data":"63599e068538a675cff33c0888285a9c8d8675478f96c62a1fedb7369c5fa5ac"}
Sep 10 21:10:30.162090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:30.162068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"853f77a9d71f48248a603af241682dcadd322d330c41f9d3a0c484a7bb320a12"}
Sep 10 21:10:30.168922 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:30.168871    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-146.ec2.internal" podStartSLOduration=20.168855339 podStartE2EDuration="20.168855339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:30.168406121 +0000 UTC m=+20.695370467" watchObservedRunningTime="2026-09-10 21:10:30.168855339 +0000 UTC m=+20.695819688"
Sep 10 21:10:30.183229 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:30.183189    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7vgf8" podStartSLOduration=1.597256181 podStartE2EDuration="20.183175809s" totalImagesPullingTime="18.585919628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:30.182796644 +0000 UTC m=+20.709760992" watchObservedRunningTime="2026-09-10 21:10:30.183175809 +0000 UTC m=+20.710140157"
Sep 10 21:10:30.955160 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:30.954934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:30.955313 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:30.955184    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:31.166731 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.166694    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g6qq8" event={"ID":"656d5e3d-d1a5-4045-b975-fc3390b46564","Type":"ContainerStarted","Data":"c3eaf759cba905468db0041076b29bb784eba983291bcfcf63c841a23bb1e009"}
Sep 10 21:10:31.170287 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.170262    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:10:31.170680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.170654    2583 generic.go:356] "Generic (PLEG): container finished" podID="7b563a9e-843d-4611-b193-7940b3016bcb" containerID="8ee0f0d3a330ad4d39265e379335f65dd1a02436a49a39abcc0cc90f1edc11cf" exitCode=1
Sep 10 21:10:31.170786 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.170725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"18a7559eb5b4b42b5761a730c49d4b63999a9fb069ded560ab1f6bc191bdece7"}
Sep 10 21:10:31.170840 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.170799    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"555e12fefbef284ca49df69482a992ec2e59e108fdf042645de0ffeabdb6e391"}
Sep 10 21:10:31.170893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.170854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"bfd0025e537c0ee91a0b5516acd5270ec08f21509affb42c2e1d17fd0f27f370"}
Sep 10 21:10:31.170930 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.170897    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"32cff7f451fd47f60f085df27f332f7bc734bd62e1e552a0b12e4a8d0021b003"}
Sep 10 21:10:31.170930 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.170925    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerDied","Data":"8ee0f0d3a330ad4d39265e379335f65dd1a02436a49a39abcc0cc90f1edc11cf"}
Sep 10 21:10:31.172389 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.172360    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs" event={"ID":"14806076-2615-4738-8dec-31341e6df82d","Type":"ContainerStarted","Data":"f93f7ee543c2ac5ae8317a0eafb258bfea5da6b6f650634dae55f31eaff7805c"}
Sep 10 21:10:31.174087 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.174063    2583 generic.go:356] "Generic (PLEG): container finished" podID="577291c3-b7fc-434f-b1a7-ceff5ec93931" containerID="d48faea9f8f09095c994c35910d397e8d96ff238e338042ce3d1c6274279d890" exitCode=0
Sep 10 21:10:31.174189 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.174150    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerDied","Data":"d48faea9f8f09095c994c35910d397e8d96ff238e338042ce3d1c6274279d890"}
Sep 10 21:10:31.175893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.175860    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6knd8" event={"ID":"437ac578-201e-4cbf-ae6d-607f2eda40a3","Type":"ContainerStarted","Data":"a0ee59e0cd9a528a2cb850f8f982402fa9074e2a302027aee167b3b34f78927e"}
Sep 10 21:10:31.177536 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.177497    2583 generic.go:356] "Generic (PLEG): container finished" podID="419cc3454d80e28154e06a9c276d2a69" containerID="9be2e32aeeb2ae798344cc9508695ae4ecbbfb074a70994facc8578f102eb8d2" exitCode=0
Sep 10 21:10:31.177623 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.177592    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal" event={"ID":"419cc3454d80e28154e06a9c276d2a69","Type":"ContainerDied","Data":"9be2e32aeeb2ae798344cc9508695ae4ecbbfb074a70994facc8578f102eb8d2"}
Sep 10 21:10:31.179235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.179134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q9ws7" event={"ID":"95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5","Type":"ContainerStarted","Data":"5236b0b86d3ed65c176deba2da62b7d9513a2b3933f1b13ce35bbdbc33ee3054"}
Sep 10 21:10:31.180879 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.180856    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l7bpv" event={"ID":"493c5fe7-fae2-47d0-9f87-55f3702df7ed","Type":"ContainerStarted","Data":"701c59769de07f81c6f3ea4fff0d8508bf943f67569c2e675f7cd8f4dcaa940f"}
Sep 10 21:10:31.181655 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.181604    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g6qq8" podStartSLOduration=2.52921451 podStartE2EDuration="21.181587112s" totalImagesPullingTime="18.652372602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:31.180861308 +0000 UTC m=+21.707825656" watchObservedRunningTime="2026-09-10 21:10:31.181587112 +0000 UTC m=+21.708551459"
Sep 10 21:10:31.218936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.218759    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q9ws7" podStartSLOduration=2.356660541 podStartE2EDuration="21.218738523s" totalImagesPullingTime="18.862077982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:31.217950061 +0000 UTC m=+21.744914409" watchObservedRunningTime="2026-09-10 21:10:31.218738523 +0000 UTC m=+21.745702870"
Sep 10 21:10:31.258466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.258416    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l7bpv" podStartSLOduration=2.598620645 podStartE2EDuration="21.258401938s" totalImagesPullingTime="18.659781293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:31.258116349 +0000 UTC m=+21.785080696" watchObservedRunningTime="2026-09-10 21:10:31.258401938 +0000 UTC m=+21.785366282"
Sep 10 21:10:31.258627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.258488    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6knd8" podStartSLOduration=6.237716554 podStartE2EDuration="21.258484047s" totalImagesPullingTime="15.020767493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:31.243814494 +0000 UTC m=+21.770778845" watchObservedRunningTime="2026-09-10 21:10:31.258484047 +0000 UTC m=+21.785448403"
Sep 10 21:10:31.367400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.367361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:31.438455 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.438413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:31.587882 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.587856    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 21:10:31.918753 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.918580    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T21:10:31.587873918Z","UUID":"222cc62c-2503-4f29-bd04-a8956d8b8751","Handler":null,"Name":"","Endpoint":""}
Sep 10 21:10:31.920227 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.920201    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 21:10:31.920227 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.920236    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 21:10:31.954640 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.954611    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:31.954810 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:31.954755    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:31.954922 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:31.954907    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:31.955033 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:31.955013    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:32.186549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.186454    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs" event={"ID":"14806076-2615-4738-8dec-31341e6df82d","Type":"ContainerStarted","Data":"31da2a455e9238eb82ec6ed619211abe2162c7624cd825721bed945c47914f0f"}
Sep 10 21:10:32.188935 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.188907    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal" event={"ID":"419cc3454d80e28154e06a9c276d2a69","Type":"ContainerStarted","Data":"257995bc9c7c6395f355b5c4e31923b26d297fce2d8f84cd7ecfaac835090a86"}
Sep 10 21:10:32.192061 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.191583    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ctbj8" event={"ID":"243623e5-602c-486d-8231-4e52706f65f9","Type":"ContainerStarted","Data":"b0fc7e433196f15114fc96f4324fa1d1adcbc7239a6a801a00f79e65334788ef"}
Sep 10 21:10:32.210208 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.210152    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-146.ec2.internal" podStartSLOduration=22.210138992 podStartE2EDuration="22.210138992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:32.209985983 +0000 UTC m=+22.736950334" watchObservedRunningTime="2026-09-10 21:10:32.210138992 +0000 UTC m=+22.737103340"
Sep 10 21:10:32.230566 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.230499    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ctbj8" podStartSLOduration=3.635893905 podStartE2EDuration="22.230484286s" totalImagesPullingTime="18.594590381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:32.23040497 +0000 UTC m=+22.757369318" watchObservedRunningTime="2026-09-10 21:10:32.230484286 +0000 UTC m=+22.757448632"
Sep 10 21:10:32.352243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.352189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:32.453685 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.453582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:32.954718 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:32.954694    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:32.954876 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:32.954807    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:33.196439 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:33.196410    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:10:33.196882 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:33.196856    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"dd52231f14e9edea9c1741b43b20d4233e0c6ccadca9a5c463dd428774f45746"}
Sep 10 21:10:33.198968 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:33.198942    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs" event={"ID":"14806076-2615-4738-8dec-31341e6df82d","Type":"ContainerStarted","Data":"8b4105b5cd216e52b9cd15f9a68b6c5d228b688230a65f592edb111663dc8845"}
Sep 10 21:10:33.218464 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:33.217549    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-scpbs" podStartSLOduration=6.402875771 podStartE2EDuration="23.217529522s" totalImagesPullingTime="16.814653751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:33.217094388 +0000 UTC m=+23.744058736" watchObservedRunningTime="2026-09-10 21:10:33.217529522 +0000 UTC m=+23.744493870"
Sep 10 21:10:33.954967 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:33.954728    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:33.955265 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:33.954734    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:33.955265 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:33.955057    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:33.955265 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:33.955159    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:34.954728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:34.954659    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:34.955121 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:34.954783    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:35.013382 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:35.013344    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:35.013999 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:35.013978    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:35.346680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:35.346591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:35.412440 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:35.412399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:35.957901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:35.957718    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:35.958296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:35.957764    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:35.958296 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:35.957998    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:35.958296 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:35.958045    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:36.209900 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.209827    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:10:36.210213 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.210183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"36c3e1f9d69f61b91981a46a3f958a7cfec1e7d8895e1411e2e4ee7f0dfa1955"}
Sep 10 21:10:36.210461 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.210435    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:36.210641 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.210626    2583 scope.go:117] "RemoveContainer" containerID="8ee0f0d3a330ad4d39265e379335f65dd1a02436a49a39abcc0cc90f1edc11cf"
Sep 10 21:10:36.212215 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.212189    2583 generic.go:356] "Generic (PLEG): container finished" podID="577291c3-b7fc-434f-b1a7-ceff5ec93931" containerID="80bd172850d9039a65c0d09e98065ddab819c06c9f5d3e3ed12b4330053ea2d2" exitCode=0
Sep 10 21:10:36.212300 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.212223    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerDied","Data":"80bd172850d9039a65c0d09e98065ddab819c06c9f5d3e3ed12b4330053ea2d2"}
Sep 10 21:10:36.234110 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.234086    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:36.954456 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:36.954421    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:36.960463 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:36.954572    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:37.121930 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.121897    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:37.219476 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.219412    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:10:37.219848 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.219823    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" event={"ID":"7b563a9e-843d-4611-b193-7940b3016bcb","Type":"ContainerStarted","Data":"6d68671a69296d583eeb3bb3ddeb2229a575b8db6344fb04df578f058e99e4a1"}
Sep 10 21:10:37.220192 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.220094    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:37.236944 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.236920    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:10:37.252120 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.252073    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m" podStartSLOduration=8.658301738 podStartE2EDuration="27.252053079s" totalImagesPullingTime="18.593751341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:37.251471223 +0000 UTC m=+27.778435581" watchObservedRunningTime="2026-09-10 21:10:37.252053079 +0000 UTC m=+27.779017418"
Sep 10 21:10:37.508488 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.508414    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkz74"]
Sep 10 21:10:37.508644 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.508618    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:37.508768 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:37.508735    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:37.511806 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.511782    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tz9tv"]
Sep 10 21:10:37.511928 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.511885    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:37.512018 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:37.511995    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:37.512432 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.512399    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7b4vh"]
Sep 10 21:10:37.512538 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:37.512497    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:37.512621 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:37.512600    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:38.224246 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:38.224210    2583 generic.go:356] "Generic (PLEG): container finished" podID="577291c3-b7fc-434f-b1a7-ceff5ec93931" containerID="910b6d88c1cb52150aa8e6a04f4de158ccc8563d2a7c1893724eb4bd8850a72a" exitCode=0
Sep 10 21:10:38.225063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:38.224291    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerDied","Data":"910b6d88c1cb52150aa8e6a04f4de158ccc8563d2a7c1893724eb4bd8850a72a"}
Sep 10 21:10:38.954957 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:38.954921    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:38.955127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:38.954996    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:38.955127 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:38.955088    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:38.955240 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:38.955194    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:39.230534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:39.230191    2583 generic.go:356] "Generic (PLEG): container finished" podID="577291c3-b7fc-434f-b1a7-ceff5ec93931" containerID="ea71f0ee05aac5408874c8b3a56c22f03fc7ea1c43b21a94564481349fa3e161" exitCode=0
Sep 10 21:10:39.230534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:39.230264    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerDied","Data":"ea71f0ee05aac5408874c8b3a56c22f03fc7ea1c43b21a94564481349fa3e161"}
Sep 10 21:10:39.689385 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:39.689351    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:39.689566 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:39.689554    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 21:10:39.690091 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:39.690068    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g6qq8"
Sep 10 21:10:39.748136 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:39.748104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:39.955540 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:39.955448    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:39.955727 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:39.955544    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:40.954157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:40.954125    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:40.954647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:40.954128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:40.954647 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:40.954231    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:40.954647 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:40.954297    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:41.008791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:41.008756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:41.109822 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:41.109785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:41.958283 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:41.958255    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:41.958717 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:41.958375    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7b4vh" podUID="f48e8a2c-2f17-4fae-ae1c-2ec72801feb2"
Sep 10 21:10:42.615976 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:42.615939    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:42.616176 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.616130    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:42.616223 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.616214    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret podName:bfd4681e-1a79-459f-b21c-118f4c0d2646 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:14.61619332 +0000 UTC m=+65.143157662 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret") pod "global-pull-secret-syncer-jkz74" (UID: "bfd4681e-1a79-459f-b21c-118f4c0d2646") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:10:42.717288 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:42.717255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:42.717440 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.717419    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:42.717525 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.717513    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:11:14.717478376 +0000 UTC m=+65.244442723 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:10:42.818004 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:42.817974    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:42.818172 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.818119    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:10:42.818172 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.818136    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:10:42.818172 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.818147    2583 projected.go:196] Error preparing data for projected volume kube-api-access-h5hrx for pod openshift-network-diagnostics/network-check-target-7b4vh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:42.818286 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.818225    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx podName:f48e8a2c-2f17-4fae-ae1c-2ec72801feb2 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:14.818208653 +0000 UTC m=+65.345172985 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h5hrx" (UniqueName: "kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx") pod "network-check-target-7b4vh" (UID: "f48e8a2c-2f17-4fae-ae1c-2ec72801feb2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:10:42.955121 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:42.955051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:42.955262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:42.955052    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:42.955262 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.955153    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz9tv" podUID="b3857789-0900-400c-b571-2a3980a3c7ee"
Sep 10 21:10:42.955262 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:42.955232    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkz74" podUID="bfd4681e-1a79-459f-b21c-118f4c0d2646"
Sep 10 21:10:43.284789 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.284723    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-146.ec2.internal" event="NodeReady"
Sep 10 21:10:43.285126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.284848    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 21:10:43.323972 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.323945    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9zgjn"]
Sep 10 21:10:43.408951 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.408917    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x9rvk"]
Sep 10 21:10:43.409108 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.409056    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.410786 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.410766    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 21:10:43.410901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.410844    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpcjb\""
Sep 10 21:10:43.410963 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.410924    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 21:10:43.432909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.432887    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9zgjn"]
Sep 10 21:10:43.432909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.432922    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x9rvk"]
Sep 10 21:10:43.433056 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.432998    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.434946 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.434928    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 21:10:43.435050 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.434966    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 21:10:43.435191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.435175    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 21:10:43.435279 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.435191    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2h9gs\""
Sep 10 21:10:43.451251 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.451220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:43.508766 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.508734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:43.521691 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.521664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf5fc50e-6872-4267-8771-2a49a4e91e71-tmp-dir\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.521823 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.521698    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpm4x\" (UniqueName: \"kubernetes.io/projected/cf5fc50e-6872-4267-8771-2a49a4e91e71-kube-api-access-tpm4x\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.521823 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.521732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5fc50e-6872-4267-8771-2a49a4e91e71-config-volume\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.521823 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.521749    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44186c88-c46e-49ea-8331-4090b76cb1d3-cert\") pod \"ingress-canary-x9rvk\" (UID: \"44186c88-c46e-49ea-8331-4090b76cb1d3\") " pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.521964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.521841    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.521964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.521872    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjgvp\" (UniqueName: \"kubernetes.io/projected/44186c88-c46e-49ea-8331-4090b76cb1d3-kube-api-access-pjgvp\") pod \"ingress-canary-x9rvk\" (UID: \"44186c88-c46e-49ea-8331-4090b76cb1d3\") " pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.623124 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623039    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf5fc50e-6872-4267-8771-2a49a4e91e71-tmp-dir\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.623124 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623098    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpm4x\" (UniqueName: \"kubernetes.io/projected/cf5fc50e-6872-4267-8771-2a49a4e91e71-kube-api-access-tpm4x\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.623344 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623156    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5fc50e-6872-4267-8771-2a49a4e91e71-config-volume\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.623344 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44186c88-c46e-49ea-8331-4090b76cb1d3-cert\") pod \"ingress-canary-x9rvk\" (UID: \"44186c88-c46e-49ea-8331-4090b76cb1d3\") " pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.623344 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.623344 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjgvp\" (UniqueName: \"kubernetes.io/projected/44186c88-c46e-49ea-8331-4090b76cb1d3-kube-api-access-pjgvp\") pod \"ingress-canary-x9rvk\" (UID: \"44186c88-c46e-49ea-8331-4090b76cb1d3\") " pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.623566 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:43.623376    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:43.623566 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:43.623438    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls podName:cf5fc50e-6872-4267-8771-2a49a4e91e71 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:44.123420588 +0000 UTC m=+34.650384927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls") pod "dns-default-9zgjn" (UID: "cf5fc50e-6872-4267-8771-2a49a4e91e71") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:43.623566 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623452    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf5fc50e-6872-4267-8771-2a49a4e91e71-tmp-dir\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.623767 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.623722    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5fc50e-6872-4267-8771-2a49a4e91e71-config-volume\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.627954 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.627932    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44186c88-c46e-49ea-8331-4090b76cb1d3-cert\") pod \"ingress-canary-x9rvk\" (UID: \"44186c88-c46e-49ea-8331-4090b76cb1d3\") " pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.631687 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.631660    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpm4x\" (UniqueName: \"kubernetes.io/projected/cf5fc50e-6872-4267-8771-2a49a4e91e71-kube-api-access-tpm4x\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:43.632190 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.632173    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjgvp\" (UniqueName: \"kubernetes.io/projected/44186c88-c46e-49ea-8331-4090b76cb1d3-kube-api-access-pjgvp\") pod \"ingress-canary-x9rvk\" (UID: \"44186c88-c46e-49ea-8331-4090b76cb1d3\") " pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.741099 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.741056    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x9rvk"
Sep 10 21:10:43.957744 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.957662    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:10:43.960016 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.959977    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:10:43.960173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.960118    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2xxbx\""
Sep 10 21:10:43.960173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:43.960145    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:10:44.127261 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:44.127226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:44.127434 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:44.127387    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:44.127479 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:44.127462    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls podName:cf5fc50e-6872-4267-8771-2a49a4e91e71 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:45.127443723 +0000 UTC m=+35.654408047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls") pod "dns-default-9zgjn" (UID: "cf5fc50e-6872-4267-8771-2a49a4e91e71") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:44.954545 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:44.954495    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:10:44.954545 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:44.954542    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:10:44.957244 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:44.957215    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:10:44.957480 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:44.957462    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xghkt\""
Sep 10 21:10:44.957594 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:44.957578    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:10:45.135254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:45.135214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:45.135430 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:45.135407    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:45.135499 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:45.135488    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls podName:cf5fc50e-6872-4267-8771-2a49a4e91e71 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:47.135468759 +0000 UTC m=+37.662433100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls") pod "dns-default-9zgjn" (UID: "cf5fc50e-6872-4267-8771-2a49a4e91e71") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:45.393100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:45.392876    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x9rvk"]
Sep 10 21:10:45.403573 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:10:45.403546    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44186c88_c46e_49ea_8331_4090b76cb1d3.slice/crio-f740e997b247cc412f0e4c4123e5792ca88f4ffa1b0b31f4c954a62a294d4dfb WatchSource:0}: Error finding container f740e997b247cc412f0e4c4123e5792ca88f4ffa1b0b31f4c954a62a294d4dfb: Status 404 returned error can't find the container with id f740e997b247cc412f0e4c4123e5792ca88f4ffa1b0b31f4c954a62a294d4dfb
Sep 10 21:10:45.967292 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:45.967185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:46.068644 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:46.068593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:46.129061 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:46.129024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:46.248166 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:46.248084    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x9rvk" event={"ID":"44186c88-c46e-49ea-8331-4090b76cb1d3","Type":"ContainerStarted","Data":"f740e997b247cc412f0e4c4123e5792ca88f4ffa1b0b31f4c954a62a294d4dfb"}
Sep 10 21:10:46.251179 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:46.251153    2583 generic.go:356] "Generic (PLEG): container finished" podID="577291c3-b7fc-434f-b1a7-ceff5ec93931" containerID="db8ebb63047ded70ca39fda1522a9cb010288c16146fdda3bc6eadf263ac7edb" exitCode=0
Sep 10 21:10:46.251305 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:46.251183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerDied","Data":"db8ebb63047ded70ca39fda1522a9cb010288c16146fdda3bc6eadf263ac7edb"}
Sep 10 21:10:46.937553 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:46.937524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:47.037459 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:47.037426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:47.151722 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:47.151684    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:47.151881 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:47.151817    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:47.151950 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:47.151883    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls podName:cf5fc50e-6872-4267-8771-2a49a4e91e71 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:51.151864171 +0000 UTC m=+41.678828497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls") pod "dns-default-9zgjn" (UID: "cf5fc50e-6872-4267-8771-2a49a4e91e71") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:47.257410 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:47.257379    2583 generic.go:356] "Generic (PLEG): container finished" podID="577291c3-b7fc-434f-b1a7-ceff5ec93931" containerID="ff5e0afdb90cc4af94c2c37f97de3aeb67e26a78c1dcb74ced59bb28077f06a9" exitCode=0
Sep 10 21:10:47.257572 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:47.257437    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerDied","Data":"ff5e0afdb90cc4af94c2c37f97de3aeb67e26a78c1dcb74ced59bb28077f06a9"}
Sep 10 21:10:48.264623 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:48.264590    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" event={"ID":"577291c3-b7fc-434f-b1a7-ceff5ec93931","Type":"ContainerStarted","Data":"7a8897eaa465fe86b43fdc95080c6a5b1b898b40cf3d5322337a0f2a459e1b45"}
Sep 10 21:10:48.304714 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:48.304652    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5d5mz" podStartSLOduration=7.283038349 podStartE2EDuration="38.304637255s" totalImagesPullingTime="31.021598906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:48.303823917 +0000 UTC m=+38.830788264" watchObservedRunningTime="2026-09-10 21:10:48.304637255 +0000 UTC m=+38.831601601"
Sep 10 21:10:48.434030 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:48.433996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:48.534363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:48.534279    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:48.617899 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:48.617868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:49.268397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:49.268365    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x9rvk" event={"ID":"44186c88-c46e-49ea-8331-4090b76cb1d3","Type":"ContainerStarted","Data":"2074708a8f0f4d44ef8f9c5a8cb32d68cc102fa850aa5363f291fc2ae963c6b4"}
Sep 10 21:10:49.282402 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:49.282349    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x9rvk" podStartSLOduration=3.510542096 podStartE2EDuration="6.282329784s" totalImagesPullingTime="2.771787688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:49.282323666 +0000 UTC m=+39.809288010" watchObservedRunningTime="2026-09-10 21:10:49.282329784 +0000 UTC m=+39.809294134"
Sep 10 21:10:51.178051 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:51.178008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:51.178442 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:51.178144    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:51.178442 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:51.178204    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls podName:cf5fc50e-6872-4267-8771-2a49a4e91e71 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:59.178188382 +0000 UTC m=+49.705152713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls") pod "dns-default-9zgjn" (UID: "cf5fc50e-6872-4267-8771-2a49a4e91e71") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:53.826494 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:53.826432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:59.240349 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:59.240309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:10:59.240782 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:59.240470    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:10:59.240782 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:10:59.240566    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls podName:cf5fc50e-6872-4267-8771-2a49a4e91e71 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:15.240547906 +0000 UTC m=+65.767512235 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls") pod "dns-default-9zgjn" (UID: "cf5fc50e-6872-4267-8771-2a49a4e91e71") : secret "dns-default-metrics-tls" not found
Sep 10 21:10:59.530533 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:59.530438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:59.583647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:10:59.583616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:02.026686 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:02.026653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:02.126705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:02.126667    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:04.143288 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:04.143249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:04.200955 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:04.200919    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:09.241289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:09.241263    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8m4m"
Sep 10 21:11:09.389913 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:09.389878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:09.446899 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:09.446870    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:14.637674 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.637636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:11:14.639839 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.639820    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:11:14.654113 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.654086    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bfd4681e-1a79-459f-b21c-118f4c0d2646-original-pull-secret\") pod \"global-pull-secret-syncer-jkz74\" (UID: \"bfd4681e-1a79-459f-b21c-118f4c0d2646\") " pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:11:14.672308 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.672280    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkz74"
Sep 10 21:11:14.738191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.738157    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:11:14.740569 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.740542    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:11:14.749053 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:14.749019    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 21:11:14.749185 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:14.749097    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs podName:b3857789-0900-400c-b571-2a3980a3c7ee nodeName:}" failed. No retries permitted until 2026-09-10 21:12:18.749075411 +0000 UTC m=+129.276039740 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs") pod "network-metrics-daemon-tz9tv" (UID: "b3857789-0900-400c-b571-2a3980a3c7ee") : secret "metrics-daemon-secret" not found
Sep 10 21:11:14.838649 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.838614    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:11:14.840665 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.840645    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:11:14.850675 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.850655    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:11:14.861810 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.861788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5hrx\" (UniqueName: \"kubernetes.io/projected/f48e8a2c-2f17-4fae-ae1c-2ec72801feb2-kube-api-access-h5hrx\") pod \"network-check-target-7b4vh\" (UID: \"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2\") " pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:11:14.870689 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.870668    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2xxbx\""
Sep 10 21:11:14.879644 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.879626    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:11:14.949762 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:14.949396    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkz74"]
Sep 10 21:11:14.954145 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:14.954102    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd4681e_1a79_459f_b21c_118f4c0d2646.slice/crio-8f1edb2afdbdec91c8b8b8165a7db8fe03af2b2c678211c2e02d26cfa29be8af WatchSource:0}: Error finding container 8f1edb2afdbdec91c8b8b8165a7db8fe03af2b2c678211c2e02d26cfa29be8af: Status 404 returned error can't find the container with id 8f1edb2afdbdec91c8b8b8165a7db8fe03af2b2c678211c2e02d26cfa29be8af
Sep 10 21:11:15.172123 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:15.172041    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7b4vh"]
Sep 10 21:11:15.175677 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:15.175649    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf48e8a2c_2f17_4fae_ae1c_2ec72801feb2.slice/crio-59aab7e44e42dc3bdadd065b86d03ec27766832c7536b5f29105e6678326848a WatchSource:0}: Error finding container 59aab7e44e42dc3bdadd065b86d03ec27766832c7536b5f29105e6678326848a: Status 404 returned error can't find the container with id 59aab7e44e42dc3bdadd065b86d03ec27766832c7536b5f29105e6678326848a
Sep 10 21:11:15.240731 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:15.240695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:11:15.240893 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:15.240854    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:11:15.240938 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:15.240928    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls podName:cf5fc50e-6872-4267-8771-2a49a4e91e71 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:47.240913339 +0000 UTC m=+97.767877663 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls") pod "dns-default-9zgjn" (UID: "cf5fc50e-6872-4267-8771-2a49a4e91e71") : secret "dns-default-metrics-tls" not found
Sep 10 21:11:15.328726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:15.328689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7b4vh" event={"ID":"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2","Type":"ContainerStarted","Data":"59aab7e44e42dc3bdadd065b86d03ec27766832c7536b5f29105e6678326848a"}
Sep 10 21:11:15.329759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:15.329736    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkz74" event={"ID":"bfd4681e-1a79-459f-b21c-118f4c0d2646","Type":"ContainerStarted","Data":"8f1edb2afdbdec91c8b8b8165a7db8fe03af2b2c678211c2e02d26cfa29be8af"}
Sep 10 21:11:20.343693 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:20.343653    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkz74" event={"ID":"bfd4681e-1a79-459f-b21c-118f4c0d2646","Type":"ContainerStarted","Data":"06baa92311d86c48997fd697dd120943b9ddf89328291e9740018187760d3e35"}
Sep 10 21:11:20.345133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:20.345109    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7b4vh" event={"ID":"f48e8a2c-2f17-4fae-ae1c-2ec72801feb2","Type":"ContainerStarted","Data":"4912dcf5babed59920feb43db450072e6cc6bfc72b6a21fb5f31eb5a25cbcddd"}
Sep 10 21:11:20.345245 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:20.345230    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:11:20.370126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:20.370075    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jkz74" podStartSLOduration=65.794472052 podStartE2EDuration="1m10.370057917s" totalImagesPullingTime="4.575585865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:20.3573208 +0000 UTC m=+70.884285147" watchObservedRunningTime="2026-09-10 21:11:20.370057917 +0000 UTC m=+70.897022261"
Sep 10 21:11:20.370681 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:20.370647    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7b4vh" podStartSLOduration=66.021750984 podStartE2EDuration="1m10.370638536s" totalImagesPullingTime="4.348887552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:20.369812491 +0000 UTC m=+70.896776828" watchObservedRunningTime="2026-09-10 21:11:20.370638536 +0000 UTC m=+70.897602884"
Sep 10 21:11:22.310697 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.310663    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"]
Sep 10 21:11:22.312553 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.312538    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.314815 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.314791    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.314815 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.314801    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 21:11:22.314815 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.314791    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 21:11:22.315028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.314813    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xlb4m\""
Sep 10 21:11:22.315028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.314801    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.317809 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.317765    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"]
Sep 10 21:11:22.319799 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.319784    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-647cb99ccb-2bwfw"]
Sep 10 21:11:22.319909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.319896    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.321684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.321660    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.321773 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.321742    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 21:11:22.321773 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.321754    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 21:11:22.321857 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.321759    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.323527 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.323495    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"]
Sep 10 21:11:22.325592 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.325573    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.325683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.325599    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vdk45\""
Sep 10 21:11:22.325960 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.325942    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 21:11:22.326027 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.325944    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 21:11:22.326254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.326233    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z68wz\""
Sep 10 21:11:22.326364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.326235    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.326364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.326295    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 21:11:22.326663 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.326647    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 21:11:22.326750 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.326666    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.334784 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.334743    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"]
Sep 10 21:11:22.335399 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.335380    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-647cb99ccb-2bwfw"]
Sep 10 21:11:22.390726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-config\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.390726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.390936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.390936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-serving-cert\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.390936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390802    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-stats-auth\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.390936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2e5032c-ac57-471c-b0c5-bca7b21236a3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.390936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc64z\" (UniqueName: \"kubernetes.io/projected/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-kube-api-access-gc64z\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.390936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390904    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-default-certificate\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.390936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390923    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gb96\" (UniqueName: \"kubernetes.io/projected/60ae789f-52b7-4762-9e54-e922b1071dfc-kube-api-access-5gb96\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.391167 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390944    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2e5032c-ac57-471c-b0c5-bca7b21236a3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.391167 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.390965    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2hps\" (UniqueName: \"kubernetes.io/projected/f2e5032c-ac57-471c-b0c5-bca7b21236a3-kube-api-access-f2hps\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.416917 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.416890    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-6ph9m"]
Sep 10 21:11:22.419027 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.419011    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.420925 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.420907    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 21:11:22.421190 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.421177    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4mxl6\""
Sep 10 21:11:22.421280 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.421259    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 21:11:22.421343 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.421282    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 21:11:22.421575 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.421561    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 21:11:22.429496 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.429472    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 21:11:22.433025 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.433006    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-6ph9m"]
Sep 10 21:11:22.491138 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:22.491284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2e5032c-ac57-471c-b0c5-bca7b21236a3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.491284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491279    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6886a0df-cdef-4730-8483-922187ca8676-service-ca-bundle\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.491419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc64z\" (UniqueName: \"kubernetes.io/projected/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-kube-api-access-gc64z\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.491419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491318    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-default-certificate\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.491419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491338    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gb96\" (UniqueName: \"kubernetes.io/projected/60ae789f-52b7-4762-9e54-e922b1071dfc-kube-api-access-5gb96\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.491419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2e5032c-ac57-471c-b0c5-bca7b21236a3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.491419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491376    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2hps\" (UniqueName: \"kubernetes.io/projected/f2e5032c-ac57-471c-b0c5-bca7b21236a3-kube-api-access-f2hps\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.491419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491392    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6886a0df-cdef-4730-8483-922187ca8676-snapshots\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-config\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6886a0df-cdef-4730-8483-922187ca8676-serving-cert\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491527    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6886a0df-cdef-4730-8483-922187ca8676-tmp\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9cql\" (UniqueName: \"kubernetes.io/projected/6886a0df-cdef-4730-8483-922187ca8676-kube-api-access-b9cql\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491611    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6886a0df-cdef-4730-8483-922187ca8676-trusted-ca-bundle\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-serving-cert\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.491726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.491717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-stats-auth\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.492152 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:22.491756    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:22.991738951 +0000 UTC m=+73.518703277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : configmap references non-existent config key: service-ca.crt
Sep 10 21:11:22.492152 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:22.492021    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 21:11:22.492152 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:22.492070    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:22.992054259 +0000 UTC m=+73.519018596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : secret "router-metrics-certs-default" not found
Sep 10 21:11:22.492152 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.492144    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-config\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.492365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.492277    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2e5032c-ac57-471c-b0c5-bca7b21236a3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.493946 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.493914    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2e5032c-ac57-471c-b0c5-bca7b21236a3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.494057 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.493977    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-serving-cert\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.494057 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.494018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-default-certificate\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.494136 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.494108    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-stats-auth\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.499311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.499264    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc64z\" (UniqueName: \"kubernetes.io/projected/5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4-kube-api-access-gc64z\") pod \"service-ca-operator-cf7c776b7-prd5j\" (UID: \"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.499638 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.499603    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2hps\" (UniqueName: \"kubernetes.io/projected/f2e5032c-ac57-471c-b0c5-bca7b21236a3-kube-api-access-f2hps\") pod \"kube-storage-version-migrator-operator-6c8f84f897-np7rq\" (UID: \"f2e5032c-ac57-471c-b0c5-bca7b21236a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.499638 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.499610    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gb96\" (UniqueName: \"kubernetes.io/projected/60ae789f-52b7-4762-9e54-e922b1071dfc-kube-api-access-5gb96\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.592479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.592376    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6886a0df-cdef-4730-8483-922187ca8676-service-ca-bundle\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.592479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.592423    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6886a0df-cdef-4730-8483-922187ca8676-snapshots\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.592479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.592451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6886a0df-cdef-4730-8483-922187ca8676-serving-cert\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.592479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.592469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6886a0df-cdef-4730-8483-922187ca8676-tmp\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.592794 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.592492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9cql\" (UniqueName: \"kubernetes.io/projected/6886a0df-cdef-4730-8483-922187ca8676-kube-api-access-b9cql\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.592794 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.592566    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6886a0df-cdef-4730-8483-922187ca8676-trusted-ca-bundle\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.593078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.593056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6886a0df-cdef-4730-8483-922187ca8676-snapshots\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.593157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.593060    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6886a0df-cdef-4730-8483-922187ca8676-tmp\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.593157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.593121    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6886a0df-cdef-4730-8483-922187ca8676-service-ca-bundle\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.593378 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.593361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6886a0df-cdef-4730-8483-922187ca8676-trusted-ca-bundle\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.594855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.594840    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6886a0df-cdef-4730-8483-922187ca8676-serving-cert\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.599952 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.599932    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9cql\" (UniqueName: \"kubernetes.io/projected/6886a0df-cdef-4730-8483-922187ca8676-kube-api-access-b9cql\") pod \"insights-operator-5844b856db-6ph9m\" (UID: \"6886a0df-cdef-4730-8483-922187ca8676\") " pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.621812 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.621785    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"
Sep 10 21:11:22.630537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.630517    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"
Sep 10 21:11:22.730327 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.730279    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-6ph9m"
Sep 10 21:11:22.891998 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.891974    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j"]
Sep 10 21:11:22.894948 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:22.894919    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c14e1f6_4baa_48ae_a3a1_f495f5e5f6d4.slice/crio-dffe98cff7249c3a56ce4235655ca8161badb25dffe2df778b4168afadf19249 WatchSource:0}: Error finding container dffe98cff7249c3a56ce4235655ca8161badb25dffe2df778b4168afadf19249: Status 404 returned error can't find the container with id dffe98cff7249c3a56ce4235655ca8161badb25dffe2df778b4168afadf19249
Sep 10 21:11:22.907306 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.907284    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq"]
Sep 10 21:11:22.910705 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:22.910682    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e5032c_ac57_471c_b0c5_bca7b21236a3.slice/crio-7dc0e8bc177afc77cc890a48f8c6e69544cd2a0fee485cf7f5a473e45e4ea8fe WatchSource:0}: Error finding container 7dc0e8bc177afc77cc890a48f8c6e69544cd2a0fee485cf7f5a473e45e4ea8fe: Status 404 returned error can't find the container with id 7dc0e8bc177afc77cc890a48f8c6e69544cd2a0fee485cf7f5a473e45e4ea8fe
Sep 10 21:11:22.995974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.995940    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.995974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:22.995980    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:22.996159 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:22.996115    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 21:11:22.996159 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:22.996123    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:23.996105187 +0000 UTC m=+74.523069512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : configmap references non-existent config key: service-ca.crt
Sep 10 21:11:22.996237 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:22.996203    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:23.996186272 +0000 UTC m=+74.523150596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : secret "router-metrics-certs-default" not found
Sep 10 21:11:23.001496 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:23.001468    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-6ph9m"]
Sep 10 21:11:23.005276 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:23.005248    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6886a0df_cdef_4730_8483_922187ca8676.slice/crio-c5eaf80b6acaa2f1c6c576e710d76ce0d8f7cfab58f5e8b27e533bf34e3d4fa3 WatchSource:0}: Error finding container c5eaf80b6acaa2f1c6c576e710d76ce0d8f7cfab58f5e8b27e533bf34e3d4fa3: Status 404 returned error can't find the container with id c5eaf80b6acaa2f1c6c576e710d76ce0d8f7cfab58f5e8b27e533bf34e3d4fa3
Sep 10 21:11:23.353561 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:23.353461    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6ph9m" event={"ID":"6886a0df-cdef-4730-8483-922187ca8676","Type":"ContainerStarted","Data":"c5eaf80b6acaa2f1c6c576e710d76ce0d8f7cfab58f5e8b27e533bf34e3d4fa3"}
Sep 10 21:11:23.354438 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:23.354414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq" event={"ID":"f2e5032c-ac57-471c-b0c5-bca7b21236a3","Type":"ContainerStarted","Data":"7dc0e8bc177afc77cc890a48f8c6e69544cd2a0fee485cf7f5a473e45e4ea8fe"}
Sep 10 21:11:23.355413 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:23.355391    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j" event={"ID":"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4","Type":"ContainerStarted","Data":"dffe98cff7249c3a56ce4235655ca8161badb25dffe2df778b4168afadf19249"}
Sep 10 21:11:23.747128 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:23.746995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:24.005029 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:24.004938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:24.005029 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:24.004985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:24.005246 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:24.005108    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 21:11:24.005246 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:24.005172    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:26.005152735 +0000 UTC m=+76.532117075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : secret "router-metrics-certs-default" not found
Sep 10 21:11:24.005603 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:24.005562    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:26.005492608 +0000 UTC m=+76.532456943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : configmap references non-existent config key: service-ca.crt
Sep 10 21:11:24.978893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:24.978845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:25.065920 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:25.065888    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:25.137875 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:25.137837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:26.020992 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.020968    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:26.021278 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.020998    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:26.021278 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:26.021107    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 21:11:26.021278 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:26.021133    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.021114766 +0000 UTC m=+80.548079108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : configmap references non-existent config key: service-ca.crt
Sep 10 21:11:26.021278 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:26.021152    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.021142507 +0000 UTC m=+80.548106837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : secret "router-metrics-certs-default" not found
Sep 10 21:11:26.371397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.371289    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j" event={"ID":"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4","Type":"ContainerStarted","Data":"3ce7f4ff93dd7d480cdd30d38c6bafc4044b713cf65d7d1c310a3a3eccbc754d"}
Sep 10 21:11:26.373113 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.373080    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6ph9m" event={"ID":"6886a0df-cdef-4730-8483-922187ca8676","Type":"ContainerStarted","Data":"c7b96af9ba9d08f48b084b4c8296bf3c51aa99865fad859b8bcd1115e8196ecf"}
Sep 10 21:11:26.374750 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.374728    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq" event={"ID":"f2e5032c-ac57-471c-b0c5-bca7b21236a3","Type":"ContainerStarted","Data":"be61d10c0b8ad57744eee41493e047c1183bbec71bba3a3d99a5d77f66981a57"}
Sep 10 21:11:26.386313 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.386261    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j" podStartSLOduration=1.257412602 podStartE2EDuration="4.386243937s" totalImagesPullingTime="3.128831335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:26.385793859 +0000 UTC m=+76.912758208" watchObservedRunningTime="2026-09-10 21:11:26.386243937 +0000 UTC m=+76.913208285"
Sep 10 21:11:26.401147 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.401090    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-6ph9m" podStartSLOduration=1.379087121 podStartE2EDuration="4.401071919s" totalImagesPullingTime="3.021984798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:26.401065085 +0000 UTC m=+76.928029424" watchObservedRunningTime="2026-09-10 21:11:26.401071919 +0000 UTC m=+76.928036266"
Sep 10 21:11:26.416430 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:26.416364    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq" podStartSLOduration=1.297720179 podStartE2EDuration="4.416344022s" totalImagesPullingTime="3.118623843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:26.415847645 +0000 UTC m=+76.942811996" watchObservedRunningTime="2026-09-10 21:11:26.416344022 +0000 UTC m=+76.943308370"
Sep 10 21:11:27.111475 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.111421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:27.120845 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.120805    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9bs6x"]
Sep 10 21:11:27.123661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.123640    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.126041 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.126019    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 21:11:27.126041 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.126030    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6nt9k\""
Sep 10 21:11:27.127203 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.127183    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 21:11:27.138851 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.138829    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9bs6x"]
Sep 10 21:11:27.231146 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.231080    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5966eddc-e221-4e73-bcf0-72728ab4a399-crio-socket\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.231293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.231153    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.231293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.231185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5966eddc-e221-4e73-bcf0-72728ab4a399-data-volume\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.231293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.231250    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.231293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.231289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx78l\" (UniqueName: \"kubernetes.io/projected/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-api-access-bx78l\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.260246 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.260218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:27.331682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.331647    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.331682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.331687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx78l\" (UniqueName: \"kubernetes.io/projected/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-api-access-bx78l\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.331909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.331771    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5966eddc-e221-4e73-bcf0-72728ab4a399-crio-socket\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.331909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.331796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.331909 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.331810    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.331909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.331823    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5966eddc-e221-4e73-bcf0-72728ab4a399-data-volume\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.331909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.331877    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5966eddc-e221-4e73-bcf0-72728ab4a399-crio-socket\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.331909 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.331881    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.332116 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.331886    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:27.831866614 +0000 UTC m=+78.358830955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.332116 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.331960    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:27.831943919 +0000 UTC m=+78.358908244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.332197 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.332150    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5966eddc-e221-4e73-bcf0-72728ab4a399-data-volume\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.339487 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.339452    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx78l\" (UniqueName: \"kubernetes.io/projected/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-api-access-bx78l\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.359467 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.359417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:27.734242 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.734208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:27.794099 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.794054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:27.835965 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.835939    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.836117 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:27.835983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:27.836117 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.836049    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.836198 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.836068    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:27.836198 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.836153    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:28.836139071 +0000 UTC m=+79.363103396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:27.836198 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:27.836195    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:28.836183806 +0000 UTC m=+79.363148131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:28.003405 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:28.003314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:28.105940 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:28.105906    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:28.216279 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:28.216248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:28.274451 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:28.274357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:28.845073 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:28.845042    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:28.845224 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:28.845096    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:28.845224 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:28.845157    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:28.845224 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:28.845214    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:28.845224 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:28.845219    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.845203 +0000 UTC m=+81.372167331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:28.845380 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:28.845263    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:30.845251658 +0000 UTC m=+81.372215983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:29.222994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.222967    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l7bpv_493c5fe7-fae2-47d0-9f87-55f3702df7ed/dns-node-resolver/0.log"
Sep 10 21:11:29.614690 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.614611    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-j4f69"]
Sep 10 21:11:29.616989 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.616970    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.618705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.618685    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 21:11:29.618815 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.618719    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 21:11:29.618958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.618941    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dmflc\""
Sep 10 21:11:29.619039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.619017    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 21:11:29.619039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.619034    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 21:11:29.627594 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.627573    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-j4f69"]
Sep 10 21:11:29.651579 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.651549    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ac41e061-37e4-4f06-bfa2-45532ae749ac-signing-key\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.651701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.651589    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w556\" (UniqueName: \"kubernetes.io/projected/ac41e061-37e4-4f06-bfa2-45532ae749ac-kube-api-access-6w556\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.651741 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.651726    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ac41e061-37e4-4f06-bfa2-45532ae749ac-signing-cabundle\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.660809 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.660781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:29.753126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.753091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ac41e061-37e4-4f06-bfa2-45532ae749ac-signing-cabundle\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.753300 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.753144    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ac41e061-37e4-4f06-bfa2-45532ae749ac-signing-key\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.753300 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.753173    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w556\" (UniqueName: \"kubernetes.io/projected/ac41e061-37e4-4f06-bfa2-45532ae749ac-kube-api-access-6w556\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.753778 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.753745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ac41e061-37e4-4f06-bfa2-45532ae749ac-signing-cabundle\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.755693 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.755674    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ac41e061-37e4-4f06-bfa2-45532ae749ac-signing-key\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.761225 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.761205    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w556\" (UniqueName: \"kubernetes.io/projected/ac41e061-37e4-4f06-bfa2-45532ae749ac-kube-api-access-6w556\") pod \"service-ca-568869fdc5-j4f69\" (UID: \"ac41e061-37e4-4f06-bfa2-45532ae749ac\") " pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:29.822256 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.822224    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6knd8_437ac578-201e-4cbf-ae6d-607f2eda40a3/node-ca/0.log"
Sep 10 21:11:29.925348 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:29.925320    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-j4f69"
Sep 10 21:11:30.055465 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.055434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:30.055653 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.055479    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:30.055653 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:30.055619    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 21:11:30.055730 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:30.055674    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:38.055660866 +0000 UTC m=+88.582625195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : secret "router-metrics-certs-default" not found
Sep 10 21:11:30.055730 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:30.055689    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:38.055682455 +0000 UTC m=+88.582646780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : configmap references non-existent config key: service-ca.crt
Sep 10 21:11:30.196918 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.196844    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-j4f69"]
Sep 10 21:11:30.200176 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:30.200149    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac41e061_37e4_4f06_bfa2_45532ae749ac.slice/crio-a8bbb74d0d0d2e3937dffaeccb0b82e63208f27ca9ce85463ec970ac36a7b72c WatchSource:0}: Error finding container a8bbb74d0d0d2e3937dffaeccb0b82e63208f27ca9ce85463ec970ac36a7b72c: Status 404 returned error can't find the container with id a8bbb74d0d0d2e3937dffaeccb0b82e63208f27ca9ce85463ec970ac36a7b72c
Sep 10 21:11:30.393546 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.393485    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-j4f69" event={"ID":"ac41e061-37e4-4f06-bfa2-45532ae749ac","Type":"ContainerStarted","Data":"ed1747a7914b23f1ca55db289e5c464d44cb8f94e36ef249cafa554b903d9993"}
Sep 10 21:11:30.394031 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.393595    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-j4f69" event={"ID":"ac41e061-37e4-4f06-bfa2-45532ae749ac","Type":"ContainerStarted","Data":"a8bbb74d0d0d2e3937dffaeccb0b82e63208f27ca9ce85463ec970ac36a7b72c"}
Sep 10 21:11:30.410374 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.410326    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-j4f69" podStartSLOduration=1.410307191 podStartE2EDuration="1.410307191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:30.409833454 +0000 UTC m=+80.936797808" watchObservedRunningTime="2026-09-10 21:11:30.410307191 +0000 UTC m=+80.937271539"
Sep 10 21:11:30.557650 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.557558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:30.861598 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.861496    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:30.861846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:30.861826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:30.862106 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:30.862090    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:30.862247 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:30.862237    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:34.862215848 +0000 UTC m=+85.389180175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:30.862375 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:30.862348    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:30.862453 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:30.862415    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:34.862398247 +0000 UTC m=+85.389362576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:31.022652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:31.022622    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x9rvk_44186c88-c46e-49ea-8331-4090b76cb1d3/serve-healthcheck-canary/0.log"
Sep 10 21:11:32.760739 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:32.760575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:33.021592 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:33.021495    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:33.118005 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:33.117974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:33.223618 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:33.223586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:33.319946 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:33.319847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:33.467688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:33.467652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:33.861255 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:33.861223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:34.464690 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:34.464656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:34.878111 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:34.878026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:34.894150 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:34.894124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:34.894263 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:34.894164    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:34.894263 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:34.894241    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:11:34.894345 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:34.894270    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:34.894345 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:34.894296    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:42.89428287 +0000 UTC m=+93.421247195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : configmap "kube-rbac-proxy" not found
Sep 10 21:11:34.894345 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:34.894312    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:42.894301765 +0000 UTC m=+93.421266090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:35.083473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:35.083439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:35.268246 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:35.268212    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:35.460991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:35.460955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.179543 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:36.179494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.464971 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:36.464877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.656236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:36.656202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:36.925623 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:36.925574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:37.055476 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:37.055430    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:37.117146 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:37.117112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:37.256905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:37.256813    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:37.456787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:37.456746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:38.120396 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:38.120355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:38.120655 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:38.120411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:38.120655 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:38.120554    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle podName:60ae789f-52b7-4762-9e54-e922b1071dfc nodeName:}" failed. No retries permitted until 2026-09-10 21:11:54.120539887 +0000 UTC m=+104.647504212 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle") pod "router-default-647cb99ccb-2bwfw" (UID: "60ae789f-52b7-4762-9e54-e922b1071dfc") : configmap references non-existent config key: service-ca.crt
Sep 10 21:11:38.122739 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:38.122714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60ae789f-52b7-4762-9e54-e922b1071dfc-metrics-certs\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:38.459073 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:38.459027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:38.867305 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:38.867219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:39.077114 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:39.077078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:39.190112 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:39.190081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:39.291136 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:39.291106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:39.471866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:39.471783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:40.267394 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:40.267359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:40.366084 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:40.366056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:40.663235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:40.663195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:41.060689 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:41.060596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:41.466719 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:41.466687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:41.666443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:41.666409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:42.276686 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:42.276650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:42.507925 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:42.507889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:42.665969 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:42.665940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:42.865206 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:42.865172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:42.962926 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:42.962843    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:42.962926 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:42.962882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:42.963101 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:42.963040    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:42.963138 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:11:42.963106    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls podName:5966eddc-e221-4e73-bcf0-72728ab4a399 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:58.963087631 +0000 UTC m=+109.490051956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9bs6x" (UID: "5966eddc-e221-4e73-bcf0-72728ab4a399") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:11:42.963471 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:42.963451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5966eddc-e221-4e73-bcf0-72728ab4a399-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:43.263166 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:43.263076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:43.470303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:43.470272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:44.424092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:44.424056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:44.548408 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:44.548370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:44.703369 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:44.703284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:44.803062 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:44.803029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:44.935284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:44.935250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:45.519442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:45.519404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:45.732109 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:45.732081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:45.911302 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:45.911269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:46.115929 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:46.115897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:46.278061 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:46.277980    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:46.379449 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:46.379414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:46.434131 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:46.434102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:47.295841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:47.295807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:11:47.297977 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:47.297958    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf5fc50e-6872-4267-8771-2a49a4e91e71-metrics-tls\") pod \"dns-default-9zgjn\" (UID: \"cf5fc50e-6872-4267-8771-2a49a4e91e71\") " pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:11:47.320752 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:47.320728    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jpcjb\""
Sep 10 21:11:47.329890 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:47.329848    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:11:47.600393 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:47.600322    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9zgjn"]
Sep 10 21:11:47.603951 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:47.603928    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5fc50e_6872_4267_8771_2a49a4e91e71.slice/crio-ac057d959b41da297fb18902357280516244afeb26bc3e07f7866e13f5d650fb WatchSource:0}: Error finding container ac057d959b41da297fb18902357280516244afeb26bc3e07f7866e13f5d650fb: Status 404 returned error can't find the container with id ac057d959b41da297fb18902357280516244afeb26bc3e07f7866e13f5d650fb
Sep 10 21:11:48.449179 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:48.449136    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zgjn" event={"ID":"cf5fc50e-6872-4267-8771-2a49a4e91e71","Type":"ContainerStarted","Data":"ac057d959b41da297fb18902357280516244afeb26bc3e07f7866e13f5d650fb"}
Sep 10 21:11:50.458329 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:50.458294    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zgjn" event={"ID":"cf5fc50e-6872-4267-8771-2a49a4e91e71","Type":"ContainerStarted","Data":"d949c8a2ec0412313cd32639675f23dc2c801927e1a64f704068885f46d2a18c"}
Sep 10 21:11:50.458755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:50.458345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9zgjn" event={"ID":"cf5fc50e-6872-4267-8771-2a49a4e91e71","Type":"ContainerStarted","Data":"c8fa413bb2f890606a4b289dfe60c61c26ad206cc0d6d465b6c5077114428a80"}
Sep 10 21:11:50.458755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:50.458424    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:11:50.480634 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:50.480581    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9zgjn" podStartSLOduration=65.116101444 podStartE2EDuration="1m7.48056439s" totalImagesPullingTime="2.364462946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:50.480019702 +0000 UTC m=+101.006984073" watchObservedRunningTime="2026-09-10 21:11:50.48056439 +0000 UTC m=+101.007528737"
Sep 10 21:11:51.350351 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:51.350321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7b4vh"
Sep 10 21:11:52.537273 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:52.537241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:52.595416 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:52.595386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:53.307688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.307655    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"]
Sep 10 21:11:53.309734 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.309719    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.312721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.312700    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 21:11:53.312721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.312703    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 21:11:53.312913 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.312774    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cxtpq\""
Sep 10 21:11:53.331652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.331623    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"]
Sep 10 21:11:53.347833 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.347807    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"]
Sep 10 21:11:53.349851 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.349830    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.353288 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.353265    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 21:11:53.353391 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.353310    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 21:11:53.353904 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.353886    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 21:11:53.354108 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.354088    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 21:11:53.354523 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.354491    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 21:11:53.355683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.355660    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 21:11:53.358730 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.358714    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 21:11:53.377602 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.377577    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"]
Sep 10 21:11:53.443554 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-hub\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.443708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/601f9feb-d42f-4071-a56c-5126b7d4709a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xcf9l\" (UID: \"601f9feb-d42f-4071-a56c-5126b7d4709a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.443708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.443708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzhcr\" (UniqueName: \"kubernetes.io/projected/2b4ab865-d283-4722-a30b-07c85fd76152-kube-api-access-vzhcr\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.443708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443632    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-ca\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.443708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b4ab865-d283-4722-a30b-07c85fd76152-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.443876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.443876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.443774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/601f9feb-d42f-4071-a56c-5126b7d4709a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xcf9l\" (UID: \"601f9feb-d42f-4071-a56c-5126b7d4709a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.471269 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.471235    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:53.545044 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.545435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/601f9feb-d42f-4071-a56c-5126b7d4709a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xcf9l\" (UID: \"601f9feb-d42f-4071-a56c-5126b7d4709a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.545435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545086    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-hub\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.545435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545110    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/601f9feb-d42f-4071-a56c-5126b7d4709a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xcf9l\" (UID: \"601f9feb-d42f-4071-a56c-5126b7d4709a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.545435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.545435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545150    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzhcr\" (UniqueName: \"kubernetes.io/projected/2b4ab865-d283-4722-a30b-07c85fd76152-kube-api-access-vzhcr\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.545435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545175    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-ca\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.545812 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b4ab865-d283-4722-a30b-07c85fd76152-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.545986 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.545964    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/601f9feb-d42f-4071-a56c-5126b7d4709a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-xcf9l\" (UID: \"601f9feb-d42f-4071-a56c-5126b7d4709a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.546382 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.546347    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b4ab865-d283-4722-a30b-07c85fd76152-additional-service-ca\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.548315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.547877    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/601f9feb-d42f-4071-a56c-5126b7d4709a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-xcf9l\" (UID: \"601f9feb-d42f-4071-a56c-5126b7d4709a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.548315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.548199    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.548315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.548287    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-hub\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.548315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.548289    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.548497 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.548292    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2b4ab865-d283-4722-a30b-07c85fd76152-ca\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.556406 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.556386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzhcr\" (UniqueName: \"kubernetes.io/projected/2b4ab865-d283-4722-a30b-07c85fd76152-kube-api-access-vzhcr\") pod \"cluster-proxy-proxy-agent-fbf549584-zvbmb\" (UID: \"2b4ab865-d283-4722-a30b-07c85fd76152\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.567549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.567485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:53.619323 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.619295    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"
Sep 10 21:11:53.672635 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.672097    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"
Sep 10 21:11:53.691155 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.678552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:53.890384 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.890354    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l"]
Sep 10 21:11:53.893339 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:53.893314    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601f9feb_d42f_4071_a56c_5126b7d4709a.slice/crio-de875e1aa261be01303f3eff600a12772b81c9cb9fc4f4b1a8c6fe7f8f9f8ce7 WatchSource:0}: Error finding container de875e1aa261be01303f3eff600a12772b81c9cb9fc4f4b1a8c6fe7f8f9f8ce7: Status 404 returned error can't find the container with id de875e1aa261be01303f3eff600a12772b81c9cb9fc4f4b1a8c6fe7f8f9f8ce7
Sep 10 21:11:53.962257 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:53.962227    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb"]
Sep 10 21:11:53.965141 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:53.965116    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b4ab865_d283_4722_a30b_07c85fd76152.slice/crio-4057abdbd611424db13003e9eda0fb78d2321b6e48192612bcbff20d8df7240a WatchSource:0}: Error finding container 4057abdbd611424db13003e9eda0fb78d2321b6e48192612bcbff20d8df7240a: Status 404 returned error can't find the container with id 4057abdbd611424db13003e9eda0fb78d2321b6e48192612bcbff20d8df7240a
Sep 10 21:11:54.150319 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:54.150231    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:54.150869 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:54.150846    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60ae789f-52b7-4762-9e54-e922b1071dfc-service-ca-bundle\") pod \"router-default-647cb99ccb-2bwfw\" (UID: \"60ae789f-52b7-4762-9e54-e922b1071dfc\") " pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:54.436516 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:54.436410    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:54.473023 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:54.472979    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb" event={"ID":"2b4ab865-d283-4722-a30b-07c85fd76152","Type":"ContainerStarted","Data":"4057abdbd611424db13003e9eda0fb78d2321b6e48192612bcbff20d8df7240a"}
Sep 10 21:11:54.474211 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:54.474189    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l" event={"ID":"601f9feb-d42f-4071-a56c-5126b7d4709a","Type":"ContainerStarted","Data":"de875e1aa261be01303f3eff600a12772b81c9cb9fc4f4b1a8c6fe7f8f9f8ce7"}
Sep 10 21:11:54.708767 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:54.708647    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-647cb99ccb-2bwfw"]
Sep 10 21:11:54.712166 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:54.712125    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ae789f_52b7_4762_9e54_e922b1071dfc.slice/crio-c12c1924b3c85a3789edacb7f5dc5b8f420d5eea8a61fe15944bec19b1ed8592 WatchSource:0}: Error finding container c12c1924b3c85a3789edacb7f5dc5b8f420d5eea8a61fe15944bec19b1ed8592: Status 404 returned error can't find the container with id c12c1924b3c85a3789edacb7f5dc5b8f420d5eea8a61fe15944bec19b1ed8592
Sep 10 21:11:55.480484 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:55.480261    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l" event={"ID":"601f9feb-d42f-4071-a56c-5126b7d4709a","Type":"ContainerStarted","Data":"6093545c7046e4d4c20fd484fb22365f7ce27f60c6636471215da84e87c03254"}
Sep 10 21:11:55.482042 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:55.482010    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-647cb99ccb-2bwfw" event={"ID":"60ae789f-52b7-4762-9e54-e922b1071dfc","Type":"ContainerStarted","Data":"46d1c7d03b1b8ba12eef8ca24d8770117230e7e06c648136f6724cefd6baf574"}
Sep 10 21:11:55.482166 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:55.482060    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-647cb99ccb-2bwfw" event={"ID":"60ae789f-52b7-4762-9e54-e922b1071dfc","Type":"ContainerStarted","Data":"c12c1924b3c85a3789edacb7f5dc5b8f420d5eea8a61fe15944bec19b1ed8592"}
Sep 10 21:11:55.494792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:55.494663    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-xcf9l" podStartSLOduration=1.151565352 podStartE2EDuration="2.494642369s" totalImagesPullingTime="1.343077017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:55.494279416 +0000 UTC m=+106.021243765" watchObservedRunningTime="2026-09-10 21:11:55.494642369 +0000 UTC m=+106.021606718"
Sep 10 21:11:55.518567 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:55.518412    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-647cb99ccb-2bwfw" podStartSLOduration=33.518391953 podStartE2EDuration="33.518391953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:55.517680369 +0000 UTC m=+106.044644707" watchObservedRunningTime="2026-09-10 21:11:55.518391953 +0000 UTC m=+106.045356307"
Sep 10 21:11:55.604765 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:55.604727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:55.702706 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:55.702624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:56.437147 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:56.437113    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:56.440293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:56.440266    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:56.486704 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:56.486670    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:56.488217 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:56.488188    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-647cb99ccb-2bwfw"
Sep 10 21:11:58.412213 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.412178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.494824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.494786    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb" event={"ID":"2b4ab865-d283-4722-a30b-07c85fd76152","Type":"ContainerStarted","Data":"f7368040930c83e071426daced29afb7de608b9cf9545ead012dbf0a8bf647a5"}
Sep 10 21:11:58.494824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.494830    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb" event={"ID":"2b4ab865-d283-4722-a30b-07c85fd76152","Type":"ContainerStarted","Data":"4bdbc68496c19b7bf3eff6a4c0dafb19281993d666369f31760d0897cb67006c"}
Sep 10 21:11:58.495067 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.494842    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb" event={"ID":"2b4ab865-d283-4722-a30b-07c85fd76152","Type":"ContainerStarted","Data":"1ac18edcb7969bbe2d7adedee76b87d86119a35f2f4daf31c34b367773a23539"}
Sep 10 21:11:58.513316 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.513268    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-fbf549584-zvbmb" podStartSLOduration=1.394547034 podStartE2EDuration="5.513253211s" totalImagesPullingTime="4.118706177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:58.513065549 +0000 UTC m=+109.040029900" watchObservedRunningTime="2026-09-10 21:11:58.513253211 +0000 UTC m=+109.040217558"
Sep 10 21:11:58.515025 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.514995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.591904 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.591864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.772476 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.772444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.993934 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.993892    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:58.996240 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:58.996216    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5966eddc-e221-4e73-bcf0-72728ab4a399-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9bs6x\" (UID: \"5966eddc-e221-4e73-bcf0-72728ab4a399\") " pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:59.233017 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:59.232959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9bs6x"
Sep 10 21:11:59.500311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:11:59.500218    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9bs6x"]
Sep 10 21:11:59.503564 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:11:59.503532    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5966eddc_e221_4e73_bcf0_72728ab4a399.slice/crio-844603d10691fd092f51ac22d5f7ea3fbc07ea15f44a8da55d9524535b170c97 WatchSource:0}: Error finding container 844603d10691fd092f51ac22d5f7ea3fbc07ea15f44a8da55d9524535b170c97: Status 404 returned error can't find the container with id 844603d10691fd092f51ac22d5f7ea3fbc07ea15f44a8da55d9524535b170c97
Sep 10 21:12:00.463720 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:00.463637    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9zgjn"
Sep 10 21:12:00.503387 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:00.503353    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9bs6x" event={"ID":"5966eddc-e221-4e73-bcf0-72728ab4a399","Type":"ContainerStarted","Data":"f84ea2812d4de901bf0154e68376a52d5021cd76725567db3a892de6ea691160"}
Sep 10 21:12:00.503753 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:00.503401    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9bs6x" event={"ID":"5966eddc-e221-4e73-bcf0-72728ab4a399","Type":"ContainerStarted","Data":"a18a33374908e8ed9cc7faa4afb0e28ba7531e414bfb70d53d9219e16df1c27f"}
Sep 10 21:12:00.503753 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:00.503421    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9bs6x" event={"ID":"5966eddc-e221-4e73-bcf0-72728ab4a399","Type":"ContainerStarted","Data":"844603d10691fd092f51ac22d5f7ea3fbc07ea15f44a8da55d9524535b170c97"}
Sep 10 21:12:01.615659 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:01.615618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:01.693371 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:01.693337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:01.941749 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:01.941710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:02.012891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:02.012856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:02.512177 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:02.512134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9bs6x" event={"ID":"5966eddc-e221-4e73-bcf0-72728ab4a399","Type":"ContainerStarted","Data":"fd3e2b70f14925da8faba1da762d3e9fc7d184003b0a76ae340d72e4571437a2"}
Sep 10 21:12:02.534230 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:02.534179    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9bs6x" podStartSLOduration=33.596772224 podStartE2EDuration="35.534163307s" totalImagesPullingTime="1.937391083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:11:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:02.533369485 +0000 UTC m=+113.060333831" watchObservedRunningTime="2026-09-10 21:12:02.534163307 +0000 UTC m=+113.061127654"
Sep 10 21:12:03.649873 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:03.649842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:03.750640 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:03.750603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:03.939398 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:03.939313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:04.041561 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:04.041524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:05.839459 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:05.839418    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:05.954279 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:05.954243    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.025632 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.025583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.076006 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.075972    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4d9cs"]
Sep 10 21:12:06.078577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.078558    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.081521 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.081485    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 21:12:06.082495 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.081717    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 21:12:06.082680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.082165    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 21:12:06.082852 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.082242    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 21:12:06.082989 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.082288    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 21:12:06.083977 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.082291    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjpkt\""
Sep 10 21:12:06.084248 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.082332    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 21:12:06.146105 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.146071    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.148496 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-textfile\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-accelerators-collector-config\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148743 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148659    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-sys\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148743 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-tls\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148743 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-wtmp\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148889 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148775    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvdjb\" (UniqueName: \"kubernetes.io/projected/cae88565-65be-44ad-9572-5008a18e891e-kube-api-access-wvdjb\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148889 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-root\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.148889 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.148835    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cae88565-65be-44ad-9572-5008a18e891e-metrics-client-ca\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249159    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-textfile\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249224    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-accelerators-collector-config\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-sys\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-tls\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249434    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-sys\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249460    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-wtmp\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvdjb\" (UniqueName: \"kubernetes.io/projected/cae88565-65be-44ad-9572-5008a18e891e-kube-api-access-wvdjb\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-root\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:12:06.249560    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cae88565-65be-44ad-9572-5008a18e891e-metrics-client-ca\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249588    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-textfile\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249997 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:12:06.249636    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-tls podName:cae88565-65be-44ad-9572-5008a18e891e nodeName:}" failed. No retries permitted until 2026-09-10 21:12:06.749615794 +0000 UTC m=+117.276580128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-tls") pod "node-exporter-4d9cs" (UID: "cae88565-65be-44ad-9572-5008a18e891e") : secret "node-exporter-tls" not found
Sep 10 21:12:06.249997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249640    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-wtmp\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.249997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.249666    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cae88565-65be-44ad-9572-5008a18e891e-root\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.250158 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.250090    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cae88565-65be-44ad-9572-5008a18e891e-metrics-client-ca\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.250158 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.250145    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-accelerators-collector-config\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.251518 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.251490    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.259424 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.259404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvdjb\" (UniqueName: \"kubernetes.io/projected/cae88565-65be-44ad-9572-5008a18e891e-kube-api-access-wvdjb\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.350579 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.350547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.446663 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.446581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.546201 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.546164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.752948 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.752860    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-tls\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.755094 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.755067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cae88565-65be-44ad-9572-5008a18e891e-node-exporter-tls\") pod \"node-exporter-4d9cs\" (UID: \"cae88565-65be-44ad-9572-5008a18e891e\") " pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:06.992284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:06.992244    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4d9cs"
Sep 10 21:12:07.003285 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:12:07.003204    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae88565_65be_44ad_9572_5008a18e891e.slice/crio-911b2be93577ba0710cf1bc89d8bd20d9b61f3e5a42d09ad3e98b067cb6f6457 WatchSource:0}: Error finding container 911b2be93577ba0710cf1bc89d8bd20d9b61f3e5a42d09ad3e98b067cb6f6457: Status 404 returned error can't find the container with id 911b2be93577ba0710cf1bc89d8bd20d9b61f3e5a42d09ad3e98b067cb6f6457
Sep 10 21:12:07.074043 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:07.074005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:07.201039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:07.201002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:07.330194 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:07.330117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:07.530695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:07.530657    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4d9cs" event={"ID":"cae88565-65be-44ad-9572-5008a18e891e","Type":"ContainerStarted","Data":"911b2be93577ba0710cf1bc89d8bd20d9b61f3e5a42d09ad3e98b067cb6f6457"}
Sep 10 21:12:07.857381 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:07.857346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:07.955176 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:07.955131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:08.374738 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:08.374635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:08.473261 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:08.473228    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:08.535936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:08.535898    2583 generic.go:356] "Generic (PLEG): container finished" podID="cae88565-65be-44ad-9572-5008a18e891e" containerID="d62b0ac98a5ce8483b22a02f71ad2831437db6cc3d2c9f013daf05fb73b259c6" exitCode=0
Sep 10 21:12:08.536129 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:08.535953    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4d9cs" event={"ID":"cae88565-65be-44ad-9572-5008a18e891e","Type":"ContainerDied","Data":"d62b0ac98a5ce8483b22a02f71ad2831437db6cc3d2c9f013daf05fb73b259c6"}
Sep 10 21:12:09.069969 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:09.069890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:09.168251 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:09.168211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:09.541850 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:09.541816    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4d9cs" event={"ID":"cae88565-65be-44ad-9572-5008a18e891e","Type":"ContainerStarted","Data":"7ac2e1649ca8de6959bee89d16b9faac33be42199df33b9eb22479cec64e38a1"}
Sep 10 21:12:09.542206 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:09.541865    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4d9cs" event={"ID":"cae88565-65be-44ad-9572-5008a18e891e","Type":"ContainerStarted","Data":"5530b72840e8542c97eaf3854582cd188ac186f18b83c4af1ea10a9eedc594d2"}
Sep 10 21:12:09.563520 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:09.563466    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4d9cs" podStartSLOduration=2.440568994 podStartE2EDuration="3.563451969s" totalImagesPullingTime="1.059630666s" totalInitContainerRuntime="63.252309ms" isStatefulPod=true podCreationTimestamp="2026-09-10 21:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:09.562726015 +0000 UTC m=+120.089690378" watchObservedRunningTime="2026-09-10 21:12:09.563451969 +0000 UTC m=+120.090416316"
Sep 10 21:12:09.632594 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:09.632552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:09.728734 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:09.728692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:10.860846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.860805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:10.864376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.864348    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"]
Sep 10 21:12:10.866952 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.866935    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:10.870101 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.870083    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qdvl7\""
Sep 10 21:12:10.870462 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.870451    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 21:12:10.877775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.877752    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"]
Sep 10 21:12:10.888638 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.888605    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1c765a24-b9fc-4e49-9275-7be899d0a311-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-52bth\" (UID: \"1c765a24-b9fc-4e49-9275-7be899d0a311\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:10.888769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.888711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkg6l\" (UniqueName: \"kubernetes.io/projected/1c765a24-b9fc-4e49-9275-7be899d0a311-kube-api-access-dkg6l\") pod \"monitoring-plugin-5d85778b9d-52bth\" (UID: \"1c765a24-b9fc-4e49-9275-7be899d0a311\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:10.898436 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.898412    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d5cddb86f-584lk"]
Sep 10 21:12:10.900311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.900298    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.902652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.902634    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 21:12:10.903152 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.903134    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 21:12:10.903345 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.903316    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 21:12:10.903442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.903365    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 21:12:10.903634 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.903611    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6bhts\""
Sep 10 21:12:10.903728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.903611    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 21:12:10.903728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.903648    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 21:12:10.906076 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.906052    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 21:12:10.911875 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.911857    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 21:12:10.914001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.913981    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d5cddb86f-584lk"]
Sep 10 21:12:10.948255 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.948222    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:10.990101 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990053    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-trusted-ca-bundle\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.990101 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990103    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-service-ca\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.990328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-config\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.990328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990224    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkg6l\" (UniqueName: \"kubernetes.io/projected/1c765a24-b9fc-4e49-9275-7be899d0a311-kube-api-access-dkg6l\") pod \"monitoring-plugin-5d85778b9d-52bth\" (UID: \"1c765a24-b9fc-4e49-9275-7be899d0a311\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:10.990328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-oauth-config\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.990328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-oauth-serving-cert\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.990328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1c765a24-b9fc-4e49-9275-7be899d0a311-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-52bth\" (UID: \"1c765a24-b9fc-4e49-9275-7be899d0a311\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:10.990328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9wxg\" (UniqueName: \"kubernetes.io/projected/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-kube-api-access-z9wxg\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.990629 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:10.990350    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-serving-cert\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:10.990629 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:12:10.990474    2583 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 21:12:10.990629 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:12:10.990574    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c765a24-b9fc-4e49-9275-7be899d0a311-monitoring-plugin-cert podName:1c765a24-b9fc-4e49-9275-7be899d0a311 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:11.490553864 +0000 UTC m=+122.017518191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/1c765a24-b9fc-4e49-9275-7be899d0a311-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-52bth" (UID: "1c765a24-b9fc-4e49-9275-7be899d0a311") : secret "monitoring-plugin-cert" not found
Sep 10 21:12:11.004441 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.004415    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkg6l\" (UniqueName: \"kubernetes.io/projected/1c765a24-b9fc-4e49-9275-7be899d0a311-kube-api-access-dkg6l\") pod \"monitoring-plugin-5d85778b9d-52bth\" (UID: \"1c765a24-b9fc-4e49-9275-7be899d0a311\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:11.091448 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.091410    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-trusted-ca-bundle\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.091448 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.091450    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-service-ca\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.091728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.091487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-config\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.091728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.091542    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-oauth-config\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.091728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.091588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-oauth-serving-cert\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.091868 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.091786    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9wxg\" (UniqueName: \"kubernetes.io/projected/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-kube-api-access-z9wxg\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.091868 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.091851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-serving-cert\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.092492 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.092465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-service-ca\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.092616 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.092549    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-config\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.092616 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.092585    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-oauth-serving-cert\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.092983 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.092952    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-trusted-ca-bundle\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.094365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.094346    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-serving-cert\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.094365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.094358    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-oauth-config\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.108895 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.108874    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9wxg\" (UniqueName: \"kubernetes.io/projected/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-kube-api-access-z9wxg\") pod \"console-d5cddb86f-584lk\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") " pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.209584 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.209544    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:11.482243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.482169    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d5cddb86f-584lk"]
Sep 10 21:12:11.485225 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:12:11.485199    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ee8737_3328_45b1_8c4b_e7a05ba5d6b0.slice/crio-08ae059a24babe4d22d3aeb8d102ebec11de957ee12a06ab3f6bfbd1aa4a64fa WatchSource:0}: Error finding container 08ae059a24babe4d22d3aeb8d102ebec11de957ee12a06ab3f6bfbd1aa4a64fa: Status 404 returned error can't find the container with id 08ae059a24babe4d22d3aeb8d102ebec11de957ee12a06ab3f6bfbd1aa4a64fa
Sep 10 21:12:11.494769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.494744    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1c765a24-b9fc-4e49-9275-7be899d0a311-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-52bth\" (UID: \"1c765a24-b9fc-4e49-9275-7be899d0a311\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:11.497223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.497197    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1c765a24-b9fc-4e49-9275-7be899d0a311-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-52bth\" (UID: \"1c765a24-b9fc-4e49-9275-7be899d0a311\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:11.517684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.517648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:11.550245 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.550218    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5cddb86f-584lk" event={"ID":"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0","Type":"ContainerStarted","Data":"08ae059a24babe4d22d3aeb8d102ebec11de957ee12a06ab3f6bfbd1aa4a64fa"}
Sep 10 21:12:11.775787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.775698    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:11.833020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.832992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:11.935735 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:11.935687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:12.045930 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:12.045858    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"]
Sep 10 21:12:12.051029 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:12:12.051001    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c765a24_b9fc_4e49_9275_7be899d0a311.slice/crio-06093307eda7c95d506d60e0b1daade305ec058603d8bde657a80538887ca4a7 WatchSource:0}: Error finding container 06093307eda7c95d506d60e0b1daade305ec058603d8bde657a80538887ca4a7: Status 404 returned error can't find the container with id 06093307eda7c95d506d60e0b1daade305ec058603d8bde657a80538887ca4a7
Sep 10 21:12:12.401106 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:12.401061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:12.555545 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:12.555497    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth" event={"ID":"1c765a24-b9fc-4e49-9275-7be899d0a311","Type":"ContainerStarted","Data":"06093307eda7c95d506d60e0b1daade305ec058603d8bde657a80538887ca4a7"}
Sep 10 21:12:12.735932 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:12.735829    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:12.837312 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:12.837261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:14.565567 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:14.565535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5cddb86f-584lk" event={"ID":"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0","Type":"ContainerStarted","Data":"b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c"}
Sep 10 21:12:14.567060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:14.567038    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth" event={"ID":"1c765a24-b9fc-4e49-9275-7be899d0a311","Type":"ContainerStarted","Data":"51cc062fcd3c462a1a1a2ad387e8d720b2a8768556bb2dd12d9eb02d596fcb92"}
Sep 10 21:12:14.567177 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:14.567081    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:14.572304 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:14.572282    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth"
Sep 10 21:12:14.582670 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:14.582623    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d5cddb86f-584lk" podStartSLOduration=1.585337543 podStartE2EDuration="4.58260803s" totalImagesPullingTime="2.997270487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:14.581633378 +0000 UTC m=+125.108597725" watchObservedRunningTime="2026-09-10 21:12:14.58260803 +0000 UTC m=+125.109572377"
Sep 10 21:12:14.595625 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:14.595494    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-52bth" podStartSLOduration=2.169906077 podStartE2EDuration="4.595476244s" totalImagesPullingTime="2.425570167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:14.59508685 +0000 UTC m=+125.122051197" watchObservedRunningTime="2026-09-10 21:12:14.595476244 +0000 UTC m=+125.122440592"
Sep 10 21:12:16.204335 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:16.204299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:16.290259 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:16.290223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:16.352542 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:16.352487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:18.516286 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:18.516240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:18.759011 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:18.758969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:12:18.761272 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:18.761253    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b3857789-0900-400c-b571-2a3980a3c7ee-metrics-certs\") pod \"network-metrics-daemon-tz9tv\" (UID: \"b3857789-0900-400c-b571-2a3980a3c7ee\") " pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:12:18.868130 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:18.868048    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xghkt\""
Sep 10 21:12:18.876549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:18.876523    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz9tv"
Sep 10 21:12:19.143833 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:19.143805    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tz9tv"]
Sep 10 21:12:19.146723 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:12:19.146696    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3857789_0900_400c_b571_2a3980a3c7ee.slice/crio-c466fa5f16bae789ae314fb06111dd649a4adc9f20516f827c802a29bdc95fc4 WatchSource:0}: Error finding container c466fa5f16bae789ae314fb06111dd649a4adc9f20516f827c802a29bdc95fc4: Status 404 returned error can't find the container with id c466fa5f16bae789ae314fb06111dd649a4adc9f20516f827c802a29bdc95fc4
Sep 10 21:12:19.588685 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:19.588601    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tz9tv" event={"ID":"b3857789-0900-400c-b571-2a3980a3c7ee","Type":"ContainerStarted","Data":"c466fa5f16bae789ae314fb06111dd649a4adc9f20516f827c802a29bdc95fc4"}
Sep 10 21:12:20.593749 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:20.593705    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tz9tv" event={"ID":"b3857789-0900-400c-b571-2a3980a3c7ee","Type":"ContainerStarted","Data":"5d4f0102c50aee86374814635402d80f92e45d24a4a6f1dd2895586c0b5164ba"}
Sep 10 21:12:20.594148 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:20.593775    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tz9tv" event={"ID":"b3857789-0900-400c-b571-2a3980a3c7ee","Type":"ContainerStarted","Data":"a9aab19634f2382d7f3d1875c3fcb70f31eb33b96603cd84ddb277f195882afe"}
Sep 10 21:12:20.609773 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:20.609726    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tz9tv" podStartSLOduration=129.704326747 podStartE2EDuration="2m10.609709863s" totalImagesPullingTime="905.383116ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:20.608436082 +0000 UTC m=+131.135400429" watchObservedRunningTime="2026-09-10 21:12:20.609709863 +0000 UTC m=+131.136674209"
Sep 10 21:12:20.697221 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:20.697191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:20.754480 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:20.754445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:21.074741 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.074711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:21.134440 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.134402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:21.210652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.210609    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:21.210780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.210675    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:21.215701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.215681    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:21.600665 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.600637    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:12:21.768541 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.768483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:21.876411 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.876320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:21.947595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:21.947558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:25.333705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:25.333668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:25.395460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:25.395424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:26.150518 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:26.150485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:26.252451 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:26.251166    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:31.277434 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:31.277398    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:31.343478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:31.343440    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:36.380041 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:36.380013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:36.481205 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:36.481167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:36.651090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:36.651061    2583 generic.go:356] "Generic (PLEG): container finished" podID="f2e5032c-ac57-471c-b0c5-bca7b21236a3" containerID="be61d10c0b8ad57744eee41493e047c1183bbec71bba3a3d99a5d77f66981a57" exitCode=0
Sep 10 21:12:36.651257 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:36.651106    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq" event={"ID":"f2e5032c-ac57-471c-b0c5-bca7b21236a3","Type":"ContainerDied","Data":"be61d10c0b8ad57744eee41493e047c1183bbec71bba3a3d99a5d77f66981a57"}
Sep 10 21:12:36.651396 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:36.651384    2583 scope.go:117] "RemoveContainer" containerID="be61d10c0b8ad57744eee41493e047c1183bbec71bba3a3d99a5d77f66981a57"
Sep 10 21:12:37.656493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:37.656457    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-np7rq" event={"ID":"f2e5032c-ac57-471c-b0c5-bca7b21236a3","Type":"ContainerStarted","Data":"43a4b8650cc42b90f22052c66002fe7aa9251e2bec486aaf061cbd823b492d7c"}
Sep 10 21:12:37.877012 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:37.876960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:37.980833 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:37.980708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:38.040637 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:38.040605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:46.273162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:46.273124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:46.328554 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:46.328490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:47.691551 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:47.691520    2583 generic.go:356] "Generic (PLEG): container finished" podID="5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4" containerID="3ce7f4ff93dd7d480cdd30d38c6bafc4044b713cf65d7d1c310a3a3eccbc754d" exitCode=0
Sep 10 21:12:47.691958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:47.691574    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j" event={"ID":"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4","Type":"ContainerDied","Data":"3ce7f4ff93dd7d480cdd30d38c6bafc4044b713cf65d7d1c310a3a3eccbc754d"}
Sep 10 21:12:47.691958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:47.691893    2583 scope.go:117] "RemoveContainer" containerID="3ce7f4ff93dd7d480cdd30d38c6bafc4044b713cf65d7d1c310a3a3eccbc754d"
Sep 10 21:12:48.697055 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:48.697022    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-prd5j" event={"ID":"5c14e1f6-4baa-48ae-a3a1-f495f5e5f6d4","Type":"ContainerStarted","Data":"2653af06479b10f494b7a956aeb02f2eacb265ae35fe1d04e349be741174758d"}
Sep 10 21:12:52.517150 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:52.517110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:52.582989 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:52.582956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:56.725887 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:56.725854    2583 generic.go:356] "Generic (PLEG): container finished" podID="6886a0df-cdef-4730-8483-922187ca8676" containerID="c7b96af9ba9d08f48b084b4c8296bf3c51aa99865fad859b8bcd1115e8196ecf" exitCode=0
Sep 10 21:12:56.726241 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:56.725913    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6ph9m" event={"ID":"6886a0df-cdef-4730-8483-922187ca8676","Type":"ContainerDied","Data":"c7b96af9ba9d08f48b084b4c8296bf3c51aa99865fad859b8bcd1115e8196ecf"}
Sep 10 21:12:56.726281 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:56.726244    2583 scope.go:117] "RemoveContainer" containerID="c7b96af9ba9d08f48b084b4c8296bf3c51aa99865fad859b8bcd1115e8196ecf"
Sep 10 21:12:57.731652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:12:57.731618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-6ph9m" event={"ID":"6886a0df-cdef-4730-8483-922187ca8676","Type":"ContainerStarted","Data":"c0ca798bbd29c216567d16eb77b75687ebe3df7e4d1d310211dcbe7418d8b8d9"}
Sep 10 21:13:07.736937 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:07.736827    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:25.412642 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:25.412594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:25.538591 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:25.538560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:33.606598 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.606558    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-747694f6dd-xc7dg"]
Sep 10 21:13:33.609771 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.609754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.623714 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.623684    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747694f6dd-xc7dg"]
Sep 10 21:13:33.630269 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.630247    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-serving-cert\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.630398 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.630278    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-config\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.630398 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.630298    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-trusted-ca-bundle\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.630398 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.630350    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-oauth-config\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.630398 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.630378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-service-ca\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.630580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.630400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6dm8\" (UniqueName: \"kubernetes.io/projected/e3148971-20f6-45e6-8fdb-0ce02dec7dac-kube-api-access-x6dm8\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.630580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.630474    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-oauth-serving-cert\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.665779 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.665737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:33.731819 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.731783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-serving-cert\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.731819 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.731825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-config\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.731845    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-trusted-ca-bundle\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.731872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-oauth-config\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.731889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-service-ca\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.731915    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6dm8\" (UniqueName: \"kubernetes.io/projected/e3148971-20f6-45e6-8fdb-0ce02dec7dac-kube-api-access-x6dm8\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.732052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-oauth-serving-cert\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732725 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.732703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-config\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732852 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.732830    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-oauth-serving-cert\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.732972 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.732949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-service-ca\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.733124 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.733109    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-trusted-ca-bundle\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.734415 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.734397    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-oauth-config\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.734528 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.734465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-serving-cert\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.739271 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.739249    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6dm8\" (UniqueName: \"kubernetes.io/projected/e3148971-20f6-45e6-8fdb-0ce02dec7dac-kube-api-access-x6dm8\") pod \"console-747694f6dd-xc7dg\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") " pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:33.768558 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.768522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:33.868015 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.867920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:33.919742 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:33.919708    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:34.190409 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:34.190379    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747694f6dd-xc7dg"]
Sep 10 21:13:34.193333 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:13:34.193310    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3148971_20f6_45e6_8fdb_0ce02dec7dac.slice/crio-80a57f8e86365fb27f712b0c93fd76c6106b1e77244a51919204ad75b1e04ff1 WatchSource:0}: Error finding container 80a57f8e86365fb27f712b0c93fd76c6106b1e77244a51919204ad75b1e04ff1: Status 404 returned error can't find the container with id 80a57f8e86365fb27f712b0c93fd76c6106b1e77244a51919204ad75b1e04ff1
Sep 10 21:13:34.415569 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:34.415536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:34.515550 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:34.515446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:34.864732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:34.864644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747694f6dd-xc7dg" event={"ID":"e3148971-20f6-45e6-8fdb-0ce02dec7dac","Type":"ContainerStarted","Data":"b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6"}
Sep 10 21:13:34.864732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:34.864692    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747694f6dd-xc7dg" event={"ID":"e3148971-20f6-45e6-8fdb-0ce02dec7dac","Type":"ContainerStarted","Data":"80a57f8e86365fb27f712b0c93fd76c6106b1e77244a51919204ad75b1e04ff1"}
Sep 10 21:13:34.885058 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:34.885008    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-747694f6dd-xc7dg" podStartSLOduration=1.884992539 podStartE2EDuration="1.884992539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:34.880704198 +0000 UTC m=+205.407668545" watchObservedRunningTime="2026-09-10 21:13:34.884992539 +0000 UTC m=+205.411956887"
Sep 10 21:13:43.920037 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:43.920002    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:43.920473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:43.920061    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:43.924749 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:43.924723    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:44.902582 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:44.902554    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:13:44.957236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:44.957194    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d5cddb86f-584lk"]
Sep 10 21:13:45.070085 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:45.070048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.135107 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:45.135075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.270186 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:45.270098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.334619 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:13:45.334587    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:09.979087 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:09.979033    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d5cddb86f-584lk" podUID="74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" containerName="console" containerID="cri-o://b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c" gracePeriod=15
Sep 10 21:14:10.215317 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.215292    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:14:10.325460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325370    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-serving-cert\") pod \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") "
Sep 10 21:14:10.325460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325424    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-oauth-config\") pod \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") "
Sep 10 21:14:10.325460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325453    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-service-ca\") pod \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") "
Sep 10 21:14:10.325758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325565    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-config\") pod \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") "
Sep 10 21:14:10.325758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325608    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-oauth-serving-cert\") pod \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") "
Sep 10 21:14:10.325758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325646    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9wxg\" (UniqueName: \"kubernetes.io/projected/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-kube-api-access-z9wxg\") pod \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") "
Sep 10 21:14:10.325758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325679    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-trusted-ca-bundle\") pod \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\" (UID: \"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0\") "
Sep 10 21:14:10.325991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325965    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-service-ca" (OuterVolumeSpecName: "service-ca") pod "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" (UID: "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:10.326056 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.325999    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-config" (OuterVolumeSpecName: "console-config") pod "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" (UID: "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:10.326122 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.326078    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" (UID: "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:10.326218 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.326186    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" (UID: "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:10.327863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.327839    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" (UID: "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:10.327959 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.327868    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-kube-api-access-z9wxg" (OuterVolumeSpecName: "kube-api-access-z9wxg") pod "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" (UID: "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0"). InnerVolumeSpecName "kube-api-access-z9wxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:10.327959 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.327923    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" (UID: "74ee8737-3328-45b1-8c4b-e7a05ba5d6b0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:10.426862 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.426827    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-trusted-ca-bundle\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:14:10.426862 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.426858    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-serving-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:14:10.426862 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.426867    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-oauth-config\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:14:10.426862 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.426876    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:14:10.427138 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.426884    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-console-config\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:14:10.427138 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.426893    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-oauth-serving-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:14:10.427138 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.426903    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9wxg\" (UniqueName: \"kubernetes.io/projected/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0-kube-api-access-z9wxg\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:14:10.993847 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.993808    2583 generic.go:356] "Generic (PLEG): container finished" podID="74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" containerID="b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c" exitCode=0
Sep 10 21:14:10.993847 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.993847    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d5cddb86f-584lk"
Sep 10 21:14:10.994303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.993892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5cddb86f-584lk" event={"ID":"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0","Type":"ContainerDied","Data":"b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c"}
Sep 10 21:14:10.994303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.993944    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d5cddb86f-584lk" event={"ID":"74ee8737-3328-45b1-8c4b-e7a05ba5d6b0","Type":"ContainerDied","Data":"08ae059a24babe4d22d3aeb8d102ebec11de957ee12a06ab3f6bfbd1aa4a64fa"}
Sep 10 21:14:10.994303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:10.993964    2583 scope.go:117] "RemoveContainer" containerID="b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c"
Sep 10 21:14:11.008086 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:11.008063    2583 scope.go:117] "RemoveContainer" containerID="b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c"
Sep 10 21:14:11.008415 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:14:11.008389    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c\": container with ID starting with b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c not found: ID does not exist" containerID="b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c"
Sep 10 21:14:11.008547 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:11.008419    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c"} err="failed to get container status \"b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c\": rpc error: code = NotFound desc = could not find container \"b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c\": container with ID starting with b93a2d5bf0027dbf574351d917b1b92ff0c519a159a350ccd47ee4fb8c51ed5c not found: ID does not exist"
Sep 10 21:14:11.019256 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:11.019229    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d5cddb86f-584lk"]
Sep 10 21:14:11.023523 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:11.023487    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d5cddb86f-584lk"]
Sep 10 21:14:11.958556 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:11.958525    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" path="/var/lib/kubelet/pods/74ee8737-3328-45b1-8c4b-e7a05ba5d6b0/volumes"
Sep 10 21:14:15.176097 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:15.176069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:15.278447 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:15.278411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:27.415418 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:27.415382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:27.474096 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:27.474062    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:40.691362 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:40.691274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:40.745174 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:40.745139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:44.634342 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.634309    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d4c6b64d6-sslfl"]
Sep 10 21:14:44.634826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.634636    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" containerName="console"
Sep 10 21:14:44.634826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.634654    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" containerName="console"
Sep 10 21:14:44.634826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.634730    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="74ee8737-3328-45b1-8c4b-e7a05ba5d6b0" containerName="console"
Sep 10 21:14:44.638312 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.638288    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.648397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.648373    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4c6b64d6-sslfl"]
Sep 10 21:14:44.675611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.675577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-console-config\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.675769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.675613    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-oauth-serving-cert\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.675769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.675644    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-oauth-config\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.675769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.675696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-serving-cert\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.675769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.675724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-service-ca\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.675769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.675759    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-trusted-ca-bundle\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.675952 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.675798    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqwzg\" (UniqueName: \"kubernetes.io/projected/a91910cc-f372-4202-be6c-bec3e601c18e-kube-api-access-qqwzg\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.693111 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.693080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:44.776174 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqwzg\" (UniqueName: \"kubernetes.io/projected/a91910cc-f372-4202-be6c-bec3e601c18e-kube-api-access-qqwzg\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.776372 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-console-config\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.776372 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-oauth-serving-cert\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.776372 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-oauth-config\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.776552 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-serving-cert\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.776552 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776417    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-service-ca\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.776552 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776466    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-trusted-ca-bundle\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.776964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.776936    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-console-config\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.777154 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.777123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-oauth-serving-cert\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.777261 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.777158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-service-ca\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.777385 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.777311    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-trusted-ca-bundle\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.778868 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.778843    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-serving-cert\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.778943 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.778928    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-oauth-config\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.784443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.784417    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqwzg\" (UniqueName: \"kubernetes.io/projected/a91910cc-f372-4202-be6c-bec3e601c18e-kube-api-access-qqwzg\") pod \"console-d4c6b64d6-sslfl\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") " pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:44.844449 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.844405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:44.947018 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:44.946977    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:45.220983 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:45.220907    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4c6b64d6-sslfl"]
Sep 10 21:14:45.224131 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:14:45.224106    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91910cc_f372_4202_be6c_bec3e601c18e.slice/crio-0a8c236b61701b20b1e8e872805457e429eddde371c39a468f40c1640eef00b5 WatchSource:0}: Error finding container 0a8c236b61701b20b1e8e872805457e429eddde371c39a468f40c1640eef00b5: Status 404 returned error can't find the container with id 0a8c236b61701b20b1e8e872805457e429eddde371c39a468f40c1640eef00b5
Sep 10 21:14:45.445984 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:45.445944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:45.501763 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:45.501681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:46.113872 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:46.113835    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4c6b64d6-sslfl" event={"ID":"a91910cc-f372-4202-be6c-bec3e601c18e","Type":"ContainerStarted","Data":"ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600"}
Sep 10 21:14:46.114239 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:46.113892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4c6b64d6-sslfl" event={"ID":"a91910cc-f372-4202-be6c-bec3e601c18e","Type":"ContainerStarted","Data":"0a8c236b61701b20b1e8e872805457e429eddde371c39a468f40c1640eef00b5"}
Sep 10 21:14:46.130722 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:46.130672    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d4c6b64d6-sslfl" podStartSLOduration=2.130657163 podStartE2EDuration="2.130657163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:14:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:46.129695852 +0000 UTC m=+276.656660200" watchObservedRunningTime="2026-09-10 21:14:46.130657163 +0000 UTC m=+276.657621510"
Sep 10 21:14:52.566971 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:52.566903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:52.621088 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:52.621054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:54.947901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:54.947860    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:54.948306 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:54.948129    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:54.952640 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:54.952620    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:55.149032 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:55.149008    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:14:55.196080 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:55.196047    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-747694f6dd-xc7dg"]
Sep 10 21:14:55.315270 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:55.315177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.415680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:55.415643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:55.472734 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:14:55.472696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:01.296832 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:01.296788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:01.399684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:01.399646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:09.876171 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:09.876136    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:15:09.876681 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:09.876464    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:15:09.884264 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:09.884243    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 21:15:20.221435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.221387    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-747694f6dd-xc7dg" podUID="e3148971-20f6-45e6-8fdb-0ce02dec7dac" containerName="console" containerID="cri-o://b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6" gracePeriod=15
Sep 10 21:15:20.465128 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.465104    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:15:20.547013 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.546930    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-service-ca\") pod \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") "
Sep 10 21:15:20.547013 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.546971    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-oauth-config\") pod \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") "
Sep 10 21:15:20.547013 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.546998    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-oauth-serving-cert\") pod \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") "
Sep 10 21:15:20.547282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547131    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-config\") pod \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") "
Sep 10 21:15:20.547282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547180    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-trusted-ca-bundle\") pod \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") "
Sep 10 21:15:20.547282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547209    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6dm8\" (UniqueName: \"kubernetes.io/projected/e3148971-20f6-45e6-8fdb-0ce02dec7dac-kube-api-access-x6dm8\") pod \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") "
Sep 10 21:15:20.547282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547260    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-serving-cert\") pod \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\" (UID: \"e3148971-20f6-45e6-8fdb-0ce02dec7dac\") "
Sep 10 21:15:20.547523 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547481    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e3148971-20f6-45e6-8fdb-0ce02dec7dac" (UID: "e3148971-20f6-45e6-8fdb-0ce02dec7dac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:20.547596 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547490    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-service-ca" (OuterVolumeSpecName: "service-ca") pod "e3148971-20f6-45e6-8fdb-0ce02dec7dac" (UID: "e3148971-20f6-45e6-8fdb-0ce02dec7dac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:20.547596 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547481    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-config" (OuterVolumeSpecName: "console-config") pod "e3148971-20f6-45e6-8fdb-0ce02dec7dac" (UID: "e3148971-20f6-45e6-8fdb-0ce02dec7dac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:20.547708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.547599    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e3148971-20f6-45e6-8fdb-0ce02dec7dac" (UID: "e3148971-20f6-45e6-8fdb-0ce02dec7dac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:20.549101 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.549080    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e3148971-20f6-45e6-8fdb-0ce02dec7dac" (UID: "e3148971-20f6-45e6-8fdb-0ce02dec7dac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:15:20.549235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.549213    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3148971-20f6-45e6-8fdb-0ce02dec7dac-kube-api-access-x6dm8" (OuterVolumeSpecName: "kube-api-access-x6dm8") pod "e3148971-20f6-45e6-8fdb-0ce02dec7dac" (UID: "e3148971-20f6-45e6-8fdb-0ce02dec7dac"). InnerVolumeSpecName "kube-api-access-x6dm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:15:20.549371 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.549355    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e3148971-20f6-45e6-8fdb-0ce02dec7dac" (UID: "e3148971-20f6-45e6-8fdb-0ce02dec7dac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:15:20.648750 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.648717    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:15:20.648750 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.648743    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-oauth-config\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:15:20.648750 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.648758    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-oauth-serving-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:15:20.648974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.648770    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-config\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:15:20.648974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.648778    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3148971-20f6-45e6-8fdb-0ce02dec7dac-trusted-ca-bundle\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:15:20.648974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.648786    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6dm8\" (UniqueName: \"kubernetes.io/projected/e3148971-20f6-45e6-8fdb-0ce02dec7dac-kube-api-access-x6dm8\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:15:20.648974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:20.648795    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3148971-20f6-45e6-8fdb-0ce02dec7dac-console-serving-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:15:21.247422 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.247387    2583 generic.go:356] "Generic (PLEG): container finished" podID="e3148971-20f6-45e6-8fdb-0ce02dec7dac" containerID="b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6" exitCode=0
Sep 10 21:15:21.247905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.247432    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747694f6dd-xc7dg"
Sep 10 21:15:21.247905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.247446    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747694f6dd-xc7dg" event={"ID":"e3148971-20f6-45e6-8fdb-0ce02dec7dac","Type":"ContainerDied","Data":"b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6"}
Sep 10 21:15:21.247905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.247484    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747694f6dd-xc7dg" event={"ID":"e3148971-20f6-45e6-8fdb-0ce02dec7dac","Type":"ContainerDied","Data":"80a57f8e86365fb27f712b0c93fd76c6106b1e77244a51919204ad75b1e04ff1"}
Sep 10 21:15:21.247905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.247525    2583 scope.go:117] "RemoveContainer" containerID="b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6"
Sep 10 21:15:21.256071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.256053    2583 scope.go:117] "RemoveContainer" containerID="b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6"
Sep 10 21:15:21.256347 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:15:21.256327    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6\": container with ID starting with b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6 not found: ID does not exist" containerID="b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6"
Sep 10 21:15:21.256411 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.256355    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6"} err="failed to get container status \"b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6\": rpc error: code = NotFound desc = could not find container \"b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6\": container with ID starting with b24f8b0624610f23ed5f17ee1993a0c55e3b303928de888149b69826a75682d6 not found: ID does not exist"
Sep 10 21:15:21.268207 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.268184    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-747694f6dd-xc7dg"]
Sep 10 21:15:21.270186 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.270165    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-747694f6dd-xc7dg"]
Sep 10 21:15:21.959204 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:21.959160    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3148971-20f6-45e6-8fdb-0ce02dec7dac" path="/var/lib/kubelet/pods/e3148971-20f6-45e6-8fdb-0ce02dec7dac/volumes"
Sep 10 21:15:25.202375 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:25.202345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:25.297193 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:25.297163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:42.598245 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:42.598209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:48.550062 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:48.550023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:51.999354 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:51.999312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:52.841479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:52.841442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:55.634611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:55.634575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:55.735196 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:55.735163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:56.722724 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:56.722682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:56.823573 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:56.823542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:57.389066 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:57.389031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:57.536519 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:57.536475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:58.942699 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:58.942663    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:59.740207 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:15:59.740171    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:02.423348 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:02.423253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:02.525187 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:02.525154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:05.978449 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:05.978411    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:06.042289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:06.042255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:07.329707 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:07.329671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:09.911570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:09.911514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:10.007584 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:10.007549    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:17.031252 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:17.031210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:17.130305 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:17.130258    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:18.841475 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:18.841442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:34.644448 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:34.644416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:35.598694 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.598651    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"]
Sep 10 21:16:35.599040 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.599017    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3148971-20f6-45e6-8fdb-0ce02dec7dac" containerName="console"
Sep 10 21:16:35.599040 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.599037    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3148971-20f6-45e6-8fdb-0ce02dec7dac" containerName="console"
Sep 10 21:16:35.599211 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.599122    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3148971-20f6-45e6-8fdb-0ce02dec7dac" containerName="console"
Sep 10 21:16:35.602077 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.602060    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.605139 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.605114    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 21:16:35.605258 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.605213    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fpp5b\""
Sep 10 21:16:35.605572 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.605550    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 21:16:35.605647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.605587    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 21:16:35.605920 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.605904    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 21:16:35.616026 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.616005    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"]
Sep 10 21:16:35.706222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.706194    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c597fcaf-3cda-4924-b2d5-dfe448498ea8-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.706222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.706226    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs5pw\" (UniqueName: \"kubernetes.io/projected/c597fcaf-3cda-4924-b2d5-dfe448498ea8-kube-api-access-zs5pw\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.706643 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.706271    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c597fcaf-3cda-4924-b2d5-dfe448498ea8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.706643 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.706340    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c597fcaf-3cda-4924-b2d5-dfe448498ea8-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.706643 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.706369    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c597fcaf-3cda-4924-b2d5-dfe448498ea8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.731289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.731254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:35.807334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.807301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c597fcaf-3cda-4924-b2d5-dfe448498ea8-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.807486 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.807342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs5pw\" (UniqueName: \"kubernetes.io/projected/c597fcaf-3cda-4924-b2d5-dfe448498ea8-kube-api-access-zs5pw\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.807486 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.807391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c597fcaf-3cda-4924-b2d5-dfe448498ea8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.807486 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.807454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c597fcaf-3cda-4924-b2d5-dfe448498ea8-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.807486 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.807482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c597fcaf-3cda-4924-b2d5-dfe448498ea8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.807871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.807847    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c597fcaf-3cda-4924-b2d5-dfe448498ea8-manifests\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.809565 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.809539    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c597fcaf-3cda-4924-b2d5-dfe448498ea8-tmp\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.809846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.809830    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c597fcaf-3cda-4924-b2d5-dfe448498ea8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.809893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.809873    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c597fcaf-3cda-4924-b2d5-dfe448498ea8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.817786 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.817767    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs5pw\" (UniqueName: \"kubernetes.io/projected/c597fcaf-3cda-4924-b2d5-dfe448498ea8-kube-api-access-zs5pw\") pod \"opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf\" (UID: \"c597fcaf-3cda-4924-b2d5-dfe448498ea8\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.831792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.831770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:35.912702 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.912667    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:35.962459 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:35.962308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:36.062258 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:36.062222    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:36.163318 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:36.163213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:36.194402 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:36.194381    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"]
Sep 10 21:16:36.196550 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:16:36.196526    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc597fcaf_3cda_4924_b2d5_dfe448498ea8.slice/crio-76c56cb30f937accc3e5d6d28041338c7295763fb891443a53ae51352da9e6ca WatchSource:0}: Error finding container 76c56cb30f937accc3e5d6d28041338c7295763fb891443a53ae51352da9e6ca: Status 404 returned error can't find the container with id 76c56cb30f937accc3e5d6d28041338c7295763fb891443a53ae51352da9e6ca
Sep 10 21:16:36.198289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:36.198273    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:16:36.519322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:36.519241    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf" event={"ID":"c597fcaf-3cda-4924-b2d5-dfe448498ea8","Type":"ContainerStarted","Data":"76c56cb30f937accc3e5d6d28041338c7295763fb891443a53ae51352da9e6ca"}
Sep 10 21:16:39.532830 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:39.532789    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf" event={"ID":"c597fcaf-3cda-4924-b2d5-dfe448498ea8","Type":"ContainerStarted","Data":"35544bf439c25f4517881de471496cb04f86bdf844bbe647ea9dccdca6a176e6"}
Sep 10 21:16:40.538322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:40.538283    2583 generic.go:356] "Generic (PLEG): container finished" podID="c597fcaf-3cda-4924-b2d5-dfe448498ea8" containerID="35544bf439c25f4517881de471496cb04f86bdf844bbe647ea9dccdca6a176e6" exitCode=0
Sep 10 21:16:40.538813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:40.538373    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf" event={"ID":"c597fcaf-3cda-4924-b2d5-dfe448498ea8","Type":"ContainerDied","Data":"35544bf439c25f4517881de471496cb04f86bdf844bbe647ea9dccdca6a176e6"}
Sep 10 21:16:41.544604 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:41.544565    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf" event={"ID":"c597fcaf-3cda-4924-b2d5-dfe448498ea8","Type":"ContainerStarted","Data":"43c6de0bb8a4156dc08997249d5c43940e2be953f69927ae97707f9e9ff97c0b"}
Sep 10 21:16:41.545085 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:41.544705    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:41.566145 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:41.566073    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf" podStartSLOduration=2.109710882 podStartE2EDuration="6.566059444s" totalImagesPullingTime="3.408421368s" totalInitContainerRuntime="1.047927194s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:41.564694292 +0000 UTC m=+392.091658639" watchObservedRunningTime="2026-09-10 21:16:41.566059444 +0000 UTC m=+392.093023790"
Sep 10 21:16:46.331223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:46.331190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:46.389596 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:46.389557    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:50.296365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:50.296332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:50.404095 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:50.404031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:52.550945 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:52.550914    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf"
Sep 10 21:16:54.785385 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:54.785351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:54.919897 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:54.919861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:55.157929 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.157893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:55.281635 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.281604    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"]
Sep 10 21:16:55.291483 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.291457    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.300101 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.300077    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-bxnvc\""
Sep 10 21:16:55.300715 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.300698    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 21:16:55.300794 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.300779    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 21:16:55.318831 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.318804    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"]
Sep 10 21:16:55.362039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.362012    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn8t5\" (UniqueName: \"kubernetes.io/projected/bfdba4f5-b266-472d-a4bf-2262c637a203-kube-api-access-zn8t5\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.362205 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.362051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfdba4f5-b266-472d-a4bf-2262c637a203-tmp\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.362205 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.362133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdba4f5-b266-472d-a4bf-2262c637a203-tls-certs\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.387310 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.387280    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:55.462572 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.462490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdba4f5-b266-472d-a4bf-2262c637a203-tls-certs\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.462722 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.462641    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn8t5\" (UniqueName: \"kubernetes.io/projected/bfdba4f5-b266-472d-a4bf-2262c637a203-kube-api-access-zn8t5\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.462722 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.462671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfdba4f5-b266-472d-a4bf-2262c637a203-tmp\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.464927 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.464905    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfdba4f5-b266-472d-a4bf-2262c637a203-tmp\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.465162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.465141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfdba4f5-b266-472d-a4bf-2262c637a203-tls-certs\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.474326 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.474301    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn8t5\" (UniqueName: \"kubernetes.io/projected/bfdba4f5-b266-472d-a4bf-2262c637a203-kube-api-access-zn8t5\") pod \"kube-auth-proxy-8545c498cb-6tj45\" (UID: \"bfdba4f5-b266-472d-a4bf-2262c637a203\") " pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.600792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.600763    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"
Sep 10 21:16:55.874939 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:55.874858    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8545c498cb-6tj45"]
Sep 10 21:16:55.877976 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:16:55.877947    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfdba4f5_b266_472d_a4bf_2262c637a203.slice/crio-983c52fc26dcddfddd0040ee73e223374bd2dc64f4decbdf3c10b22b18ccc145 WatchSource:0}: Error finding container 983c52fc26dcddfddd0040ee73e223374bd2dc64f4decbdf3c10b22b18ccc145: Status 404 returned error can't find the container with id 983c52fc26dcddfddd0040ee73e223374bd2dc64f4decbdf3c10b22b18ccc145
Sep 10 21:16:56.602076 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:56.602040    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45" event={"ID":"bfdba4f5-b266-472d-a4bf-2262c637a203","Type":"ContainerStarted","Data":"983c52fc26dcddfddd0040ee73e223374bd2dc64f4decbdf3c10b22b18ccc145"}
Sep 10 21:16:56.867337 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:56.867217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:56.941410 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:56.941371    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:57.828141 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:57.828096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:59.616984 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:59.616945    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45" event={"ID":"bfdba4f5-b266-472d-a4bf-2262c637a203","Type":"ContainerStarted","Data":"5d425e9ea3cda64da6727e16bd73e7230130e55e82b7f975759694679bc2a3ab"}
Sep 10 21:16:59.633116 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:16:59.633067    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8545c498cb-6tj45" podStartSLOduration=1.9179842360000001 podStartE2EDuration="4.633051499s" totalImagesPullingTime="2.715067263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:59.631323964 +0000 UTC m=+410.158288311" watchObservedRunningTime="2026-09-10 21:16:59.633051499 +0000 UTC m=+410.160015845"
Sep 10 21:17:02.331353 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.331320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:02.431775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.431736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:02.587490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.587422    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"]
Sep 10 21:17:02.589993 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.589972    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.592868 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.592843    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fxntm\""
Sep 10 21:17:02.596262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.596238    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:17:02.619288 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.619256    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"]
Sep 10 21:17:02.663895 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.663858    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:02.726805 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.726762    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"]
Sep 10 21:17:02.727089 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727065    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727139    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727264 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727264 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727232    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2254a779-b871-4af2-8530-bf35f67ce8a2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727264 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727257    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727415 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727280    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727415 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727349    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.727415 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.727390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47hpl\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-kube-api-access-47hpl\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.729215 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.729194    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.754862 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.754834    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"]
Sep 10 21:17:02.788132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.788098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:02.828523 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828468    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.828698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.828698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.828698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828597    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.828698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828623    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls4hb\" (UniqueName: \"kubernetes.io/projected/7d586b02-0c0b-494c-a319-cf4f777ba63d-kube-api-access-ls4hb\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.828698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.828698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828686    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47hpl\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-kube-api-access-47hpl\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.828975 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828770    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.828975 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.828975 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828877    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d586b02-0c0b-494c-a319-cf4f777ba63d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.828975 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.828975 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.828975 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.829276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.828988    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829012    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.829276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829080    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829153    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829203    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2254a779-b871-4af2-8530-bf35f67ce8a2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829201    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829657 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829389    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829657 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.829906 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.829883    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2254a779-b871-4af2-8530-bf35f67ce8a2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.830964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.830937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.831222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.831202    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.845468 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.845406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.849052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.849031    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47hpl\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-kube-api-access-47hpl\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.884249 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.884212    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:02.901100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.901071    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:02.930659 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.930629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d586b02-0c0b-494c-a319-cf4f777ba63d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.930807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.930666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.930807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.930683    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.930919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.930801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.930919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.930850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931098 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931173    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls4hb\" (UniqueName: \"kubernetes.io/projected/7d586b02-0c0b-494c-a319-cf4f777ba63d-kube-api-access-ls4hb\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931267 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931178    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931325 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931419    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931449    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d586b02-0c0b-494c-a319-cf4f777ba63d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.931630 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.931615    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.933175 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.933155    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.933533 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.933494    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.939381 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.939356    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d586b02-0c0b-494c-a319-cf4f777ba63d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:02.939775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:02.939753    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls4hb\" (UniqueName: \"kubernetes.io/projected/7d586b02-0c0b-494c-a319-cf4f777ba63d-kube-api-access-ls4hb\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7\" (UID: \"7d586b02-0c0b-494c-a319-cf4f777ba63d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:03.038992 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:03.038958    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:03.175392 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:03.175362    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"]
Sep 10 21:17:03.177919 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:17:03.177894    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2254a779_b871_4af2_8530_bf35f67ce8a2.slice/crio-e6f6d3872f0cc4bbf16dd1d6d228407d942ce90d4f81d5b03085d4bd9eabc1b4 WatchSource:0}: Error finding container e6f6d3872f0cc4bbf16dd1d6d228407d942ce90d4f81d5b03085d4bd9eabc1b4: Status 404 returned error can't find the container with id e6f6d3872f0cc4bbf16dd1d6d228407d942ce90d4f81d5b03085d4bd9eabc1b4
Sep 10 21:17:03.310527 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:03.310487    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"]
Sep 10 21:17:03.312583 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:17:03.312546    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d586b02_0c0b_494c_a319_cf4f777ba63d.slice/crio-8337aa29f6b635cd5774acf07e1b5851b643478a7857836b82594c4e1e0e25e3 WatchSource:0}: Error finding container 8337aa29f6b635cd5774acf07e1b5851b643478a7857836b82594c4e1e0e25e3: Status 404 returned error can't find the container with id 8337aa29f6b635cd5774acf07e1b5851b643478a7857836b82594c4e1e0e25e3
Sep 10 21:17:03.633830 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:03.633739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7" event={"ID":"7d586b02-0c0b-494c-a319-cf4f777ba63d","Type":"ContainerStarted","Data":"8337aa29f6b635cd5774acf07e1b5851b643478a7857836b82594c4e1e0e25e3"}
Sep 10 21:17:03.634890 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:03.634865    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" event={"ID":"2254a779-b871-4af2-8530-bf35f67ce8a2","Type":"ContainerStarted","Data":"e6f6d3872f0cc4bbf16dd1d6d228407d942ce90d4f81d5b03085d4bd9eabc1b4"}
Sep 10 21:17:04.848158 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.847902    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9589c846d-qgkqn"]
Sep 10 21:17:04.851705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.851487    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:04.862704 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.862676    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9589c846d-qgkqn"]
Sep 10 21:17:04.907728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.907695    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:04.952960 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.952912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-trusted-ca-bundle\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:04.952960 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.952965    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7fvh\" (UniqueName: \"kubernetes.io/projected/270a1aa7-f318-48ba-afd8-32e2efd47e02-kube-api-access-m7fvh\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:04.953180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.952987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-service-ca\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:04.953180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.953030    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-oauth-config\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:04.953180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.953059    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-oauth-serving-cert\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:04.953180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.953101    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-serving-cert\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:04.953180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:04.953162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-config\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.040516 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.040464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.054487 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.054451    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-serving-cert\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.054683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.054548    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-config\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.054683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.054604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-trusted-ca-bundle\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.054683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.054634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7fvh\" (UniqueName: \"kubernetes.io/projected/270a1aa7-f318-48ba-afd8-32e2efd47e02-kube-api-access-m7fvh\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.054839 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.054805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-service-ca\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.054908 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.054890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-oauth-config\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.054962 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.054922    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-oauth-serving-cert\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.055353 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.055309    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-config\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.055758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.055729    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-service-ca\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.055871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.055808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-oauth-serving-cert\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.055871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.055818    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/270a1aa7-f318-48ba-afd8-32e2efd47e02-trusted-ca-bundle\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.057458 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.057439    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-serving-cert\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.057712 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.057690    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/270a1aa7-f318-48ba-afd8-32e2efd47e02-console-oauth-config\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.065365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.065342    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7fvh\" (UniqueName: \"kubernetes.io/projected/270a1aa7-f318-48ba-afd8-32e2efd47e02-kube-api-access-m7fvh\") pod \"console-9589c846d-qgkqn\" (UID: \"270a1aa7-f318-48ba-afd8-32e2efd47e02\") " pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.102176 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.102076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.164295 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.164258    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:05.642316 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.642278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.742577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.742540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.859560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.859526    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b52gc"]
Sep 10 21:17:05.867797 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.867774    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:05.873178 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.873155    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 10 21:17:05.873342 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.873321    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-znjm6\""
Sep 10 21:17:05.882708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.882636    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b52gc"]
Sep 10 21:17:05.908748 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.908715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.965014 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.964982    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fde57b2-669a-442d-8f08-12ea52ef260f-cert\") pod \"odh-model-controller-7998c67dc4-b52gc\" (UID: \"8fde57b2-669a-442d-8f08-12ea52ef260f\") " pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:05.965187 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.965122    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vks8b\" (UniqueName: \"kubernetes.io/projected/8fde57b2-669a-442d-8f08-12ea52ef260f-kube-api-access-vks8b\") pod \"odh-model-controller-7998c67dc4-b52gc\" (UID: \"8fde57b2-669a-442d-8f08-12ea52ef260f\") " pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:05.995437 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.995377    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:05.995631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.995482    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:05.995631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:05.995568    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:06.002645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.002494    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:06.002645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.002599    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:06.002732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.002661    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:06.003279 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.003251    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:06.066620 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.066379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vks8b\" (UniqueName: \"kubernetes.io/projected/8fde57b2-669a-442d-8f08-12ea52ef260f-kube-api-access-vks8b\") pod \"odh-model-controller-7998c67dc4-b52gc\" (UID: \"8fde57b2-669a-442d-8f08-12ea52ef260f\") " pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:06.066620 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.066488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fde57b2-669a-442d-8f08-12ea52ef260f-cert\") pod \"odh-model-controller-7998c67dc4-b52gc\" (UID: \"8fde57b2-669a-442d-8f08-12ea52ef260f\") " pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:06.067039 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:17:06.066897    2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 10 21:17:06.067039 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:17:06.066976    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fde57b2-669a-442d-8f08-12ea52ef260f-cert podName:8fde57b2-669a-442d-8f08-12ea52ef260f nodeName:}" failed. No retries permitted until 2026-09-10 21:17:06.566946562 +0000 UTC m=+417.093910890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8fde57b2-669a-442d-8f08-12ea52ef260f-cert") pod "odh-model-controller-7998c67dc4-b52gc" (UID: "8fde57b2-669a-442d-8f08-12ea52ef260f") : secret "odh-model-controller-webhook-cert" not found
Sep 10 21:17:06.078100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.078070    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vks8b\" (UniqueName: \"kubernetes.io/projected/8fde57b2-669a-442d-8f08-12ea52ef260f-kube-api-access-vks8b\") pod \"odh-model-controller-7998c67dc4-b52gc\" (UID: \"8fde57b2-669a-442d-8f08-12ea52ef260f\") " pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:06.169025 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.168933    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9589c846d-qgkqn"]
Sep 10 21:17:06.174316 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:17:06.174256    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270a1aa7_f318_48ba_afd8_32e2efd47e02.slice/crio-c54be795c38b755bd7d929ea2fa862371c727b547d0c42490c80d9fe3797464c WatchSource:0}: Error finding container c54be795c38b755bd7d929ea2fa862371c727b547d0c42490c80d9fe3797464c: Status 404 returned error can't find the container with id c54be795c38b755bd7d929ea2fa862371c727b547d0c42490c80d9fe3797464c
Sep 10 21:17:06.190293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.190225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:06.458153 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.458063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:06.573760 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.573722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fde57b2-669a-442d-8f08-12ea52ef260f-cert\") pod \"odh-model-controller-7998c67dc4-b52gc\" (UID: \"8fde57b2-669a-442d-8f08-12ea52ef260f\") " pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:06.576303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.576278    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fde57b2-669a-442d-8f08-12ea52ef260f-cert\") pod \"odh-model-controller-7998c67dc4-b52gc\" (UID: \"8fde57b2-669a-442d-8f08-12ea52ef260f\") " pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:06.653322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.653236    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9589c846d-qgkqn" event={"ID":"270a1aa7-f318-48ba-afd8-32e2efd47e02","Type":"ContainerStarted","Data":"70f6c5fd9ca86d7f8f704edcff38d3f9a87aa532ad8ef076978783d4e6115e97"}
Sep 10 21:17:06.653322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.653302    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9589c846d-qgkqn" event={"ID":"270a1aa7-f318-48ba-afd8-32e2efd47e02","Type":"ContainerStarted","Data":"c54be795c38b755bd7d929ea2fa862371c727b547d0c42490c80d9fe3797464c"}
Sep 10 21:17:06.655303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.655253    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7" event={"ID":"7d586b02-0c0b-494c-a319-cf4f777ba63d","Type":"ContainerStarted","Data":"1e6203b504684bb3bc811da01feefdff8753aaeb19dcb6bb0b2804ef3c2cf45a"}
Sep 10 21:17:06.656824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.656801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" event={"ID":"2254a779-b871-4af2-8530-bf35f67ce8a2","Type":"ContainerStarted","Data":"ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f"}
Sep 10 21:17:06.670001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.669960    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9589c846d-qgkqn" podStartSLOduration=2.669945686 podStartE2EDuration="2.669945686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:06.669354769 +0000 UTC m=+417.196319121" watchObservedRunningTime="2026-09-10 21:17:06.669945686 +0000 UTC m=+417.196910033"
Sep 10 21:17:06.687046 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.686999    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7" podStartSLOduration=1.9994633130000001 podStartE2EDuration="4.686980117s" totalImagesPullingTime="2.687516804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:06.686294754 +0000 UTC m=+417.213259100" watchObservedRunningTime="2026-09-10 21:17:06.686980117 +0000 UTC m=+417.213944468"
Sep 10 21:17:06.705251 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.705197    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" podStartSLOduration=1.89001318 podStartE2EDuration="4.705176342s" totalImagesPullingTime="2.815163162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:06.704820056 +0000 UTC m=+417.231784405" watchObservedRunningTime="2026-09-10 21:17:06.705176342 +0000 UTC m=+417.232140691"
Sep 10 21:17:06.769545 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.769442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:06.780294 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.780270    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:06.901784 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:06.901749    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:07.040060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.039972    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:07.045206 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.045181    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:07.052785 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.052755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b52gc"]
Sep 10 21:17:07.056219 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:17:07.056192    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fde57b2_669a_442d_8f08_12ea52ef260f.slice/crio-3016af0a3244a356f1e567a35c52d5ca348f29fbc4c51a375b2ccd7d936e92c9 WatchSource:0}: Error finding container 3016af0a3244a356f1e567a35c52d5ca348f29fbc4c51a375b2ccd7d936e92c9: Status 404 returned error can't find the container with id 3016af0a3244a356f1e567a35c52d5ca348f29fbc4c51a375b2ccd7d936e92c9
Sep 10 21:17:07.662386 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.662351    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b52gc" event={"ID":"8fde57b2-669a-442d-8f08-12ea52ef260f","Type":"ContainerStarted","Data":"3016af0a3244a356f1e567a35c52d5ca348f29fbc4c51a375b2ccd7d936e92c9"}
Sep 10 21:17:07.662803 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.662778    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:07.663835 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.663815    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7"
Sep 10 21:17:07.720083 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.720037    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"]
Sep 10 21:17:07.903230 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.903189    2583 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body=
Sep 10 21:17:07.903708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:07.903269    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" podUID="2254a779-b871-4af2-8530-bf35f67ce8a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused"
Sep 10 21:17:08.902136 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:08.902099    2583 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body=
Sep 10 21:17:08.902286 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:08.902159    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" podUID="2254a779-b871-4af2-8530-bf35f67ce8a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused"
Sep 10 21:17:09.679068 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:09.675749    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" podUID="2254a779-b871-4af2-8530-bf35f67ce8a2" containerName="istio-proxy" containerID="cri-o://ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f" gracePeriod=30
Sep 10 21:17:10.681549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:10.681515    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b52gc" event={"ID":"8fde57b2-669a-442d-8f08-12ea52ef260f","Type":"ContainerStarted","Data":"f51a259ebd9e30db0351ecf0a8f0d6ba40107b1f92afd62b2206d1ea28c3937b"}
Sep 10 21:17:10.681913 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:10.681702    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:10.697442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:10.697391    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-b52gc" podStartSLOduration=2.9107913659999998 podStartE2EDuration="5.697374256s" totalImagesPullingTime="2.78658289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:10.696409306 +0000 UTC m=+421.223373653" watchObservedRunningTime="2026-09-10 21:17:10.697374256 +0000 UTC m=+421.224338642"
Sep 10 21:17:13.391916 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.391884    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"]
Sep 10 21:17:13.395175 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.395158    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.397066 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.397041    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 21:17:13.397216 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.397067    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 21:17:13.397216 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.397043    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 21:17:13.397216 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.397113    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nxgtr\""
Sep 10 21:17:13.408100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.408076    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"]
Sep 10 21:17:13.437510 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.437469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.437627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.437524    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n87p2\" (UniqueName: \"kubernetes.io/projected/b735a53a-1cf4-48bd-8926-21d406a91b0c-kube-api-access-n87p2\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.437627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.437602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.437730 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.437635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b735a53a-1cf4-48bd-8926-21d406a91b0c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.437730 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.437655    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.437730 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.437673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.437859 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.437765    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.538494 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.538455    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87p2\" (UniqueName: \"kubernetes.io/projected/b735a53a-1cf4-48bd-8926-21d406a91b0c-kube-api-access-n87p2\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.538682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.538545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.538682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.538581    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b735a53a-1cf4-48bd-8926-21d406a91b0c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.538682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.538608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.538682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.538639    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.538873 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.538694    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.538873 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.538738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.539582 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.539548    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.541081 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.541057    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.541182 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.541134    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b735a53a-1cf4-48bd-8926-21d406a91b0c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.541221 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.541210    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.541257 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.541228    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.553617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.553599    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b735a53a-1cf4-48bd-8926-21d406a91b0c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.553861 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.553844    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87p2\" (UniqueName: \"kubernetes.io/projected/b735a53a-1cf4-48bd-8926-21d406a91b0c-kube-api-access-n87p2\") pod \"istiod-openshift-gateway-5677ccb49b-4gb5k\" (UID: \"b735a53a-1cf4-48bd-8926-21d406a91b0c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.703723 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.703636    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:13.977628 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:13.977548    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"]
Sep 10 21:17:13.981319 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:17:13.981296    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb735a53a_1cf4_48bd_8926_21d406a91b0c.slice/crio-9385f3b83053110818f6e43628e813ade631381b499628cb10e610f08acf90ae WatchSource:0}: Error finding container 9385f3b83053110818f6e43628e813ade631381b499628cb10e610f08acf90ae: Status 404 returned error can't find the container with id 9385f3b83053110818f6e43628e813ade631381b499628cb10e610f08acf90ae
Sep 10 21:17:14.700409 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:14.700381    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k" event={"ID":"b735a53a-1cf4-48bd-8926-21d406a91b0c","Type":"ContainerStarted","Data":"9385f3b83053110818f6e43628e813ade631381b499628cb10e610f08acf90ae"}
Sep 10 21:17:14.953330 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:14.953269    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:15.051658 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051624    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-envoy\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.051843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051672    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-data\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.051843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051699    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-certs\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.051843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051739    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-podinfo\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.052010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051847    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-socket\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.052010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051906    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47hpl\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-kube-api-access-47hpl\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.052010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051959    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2254a779-b871-4af2-8530-bf35f67ce8a2-istiod-ca-cert\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.052163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.051982    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:17:15.052163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052042    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:17:15.052163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052065    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-token\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.052163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052104    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-credential-socket\") pod \"2254a779-b871-4af2-8530-bf35f67ce8a2\" (UID: \"2254a779-b871-4af2-8530-bf35f67ce8a2\") "
Sep 10 21:17:15.052370 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052158    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:17:15.052425 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052386    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:17:15.052425 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052409    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2254a779-b871-4af2-8530-bf35f67ce8a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:15.052562 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052418    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.052562 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052455    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-workload-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.052562 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.052470    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-data\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.054570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.054535    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:15.054570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.054551    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:17:15.054747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.054609    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:17:15.054747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.054719    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-kube-api-access-47hpl" (OuterVolumeSpecName: "kube-api-access-47hpl") pod "2254a779-b871-4af2-8530-bf35f67ce8a2" (UID: "2254a779-b871-4af2-8530-bf35f67ce8a2"). InnerVolumeSpecName "kube-api-access-47hpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:15.153005 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.152970    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-token\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.153005 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.153005    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-credential-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.153225 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.153020    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-envoy\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.153225 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.153035    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2254a779-b871-4af2-8530-bf35f67ce8a2-istio-podinfo\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.153225 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.153047    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47hpl\" (UniqueName: \"kubernetes.io/projected/2254a779-b871-4af2-8530-bf35f67ce8a2-kube-api-access-47hpl\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.153225 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.153055    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2254a779-b871-4af2-8530-bf35f67ce8a2-istiod-ca-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:15.164940 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.164903    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:15.165106 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.164980    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:15.170487 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.170461    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:15.707347 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.707309    2583 generic.go:356] "Generic (PLEG): container finished" podID="2254a779-b871-4af2-8530-bf35f67ce8a2" containerID="ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f" exitCode=0
Sep 10 21:17:15.707843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.707368    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"
Sep 10 21:17:15.707843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.707396    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" event={"ID":"2254a779-b871-4af2-8530-bf35f67ce8a2","Type":"ContainerDied","Data":"ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f"}
Sep 10 21:17:15.707843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.707458    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m" event={"ID":"2254a779-b871-4af2-8530-bf35f67ce8a2","Type":"ContainerDied","Data":"e6f6d3872f0cc4bbf16dd1d6d228407d942ce90d4f81d5b03085d4bd9eabc1b4"}
Sep 10 21:17:15.707843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.707484    2583 scope.go:117] "RemoveContainer" containerID="ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f"
Sep 10 21:17:15.713009 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.712940    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9589c846d-qgkqn"
Sep 10 21:17:15.753026 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.752995    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"]
Sep 10 21:17:15.756049 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.756022    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc5kr7m"]
Sep 10 21:17:15.766871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.766792    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d4c6b64d6-sslfl"]
Sep 10 21:17:15.961233 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.961158    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2254a779-b871-4af2-8530-bf35f67ce8a2" path="/var/lib/kubelet/pods/2254a779-b871-4af2-8530-bf35f67ce8a2/volumes"
Sep 10 21:17:15.983321 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:15.983291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:16.115158 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.115125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:16.478076 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.478043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:16.488825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.488729    2583 scope.go:117] "RemoveContainer" containerID="ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f"
Sep 10 21:17:16.490143 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:17:16.490104    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f\": container with ID starting with ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f not found: ID does not exist" containerID="ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f"
Sep 10 21:17:16.490288 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.490144    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f"} err="failed to get container status \"ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f\": rpc error: code = NotFound desc = could not find container \"ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f\": container with ID starting with ace1c9ae6209a8957ce299fd9a70515e7b8f662be28de9a249c22d603eed0c9f not found: ID does not exist"
Sep 10 21:17:16.548589 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.548553    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:16.548682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.548639    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:17:16.614020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.613953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:16.713519 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.713469    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k" event={"ID":"b735a53a-1cf4-48bd-8926-21d406a91b0c","Type":"ContainerStarted","Data":"91d8a3edc625d3c7547d0869b2178dd441956f626e2c9b418697ff839d6a63ef"}
Sep 10 21:17:16.713937 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.713548    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:16.766359 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.766239    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k" podStartSLOduration=1.201258226 podStartE2EDuration="3.766217771s" totalImagesPullingTime="2.564959545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:16.765159076 +0000 UTC m=+427.292123423" watchObservedRunningTime="2026-09-10 21:17:16.766217771 +0000 UTC m=+427.293182122"
Sep 10 21:17:16.766941 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.766889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:16.863625 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:16.863589    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:17.720695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:17.720663    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-4gb5k container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:17:17.721077 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:17.720711    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k" podUID="b735a53a-1cf4-48bd-8926-21d406a91b0c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:17:17.891550 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:17.891497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:18.016185 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:18.016086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:18.109178 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:18.109144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:20.719918 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:20.719885    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-4gb5k"
Sep 10 21:17:21.689623 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:21.689594    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b52gc"
Sep 10 21:17:40.791304 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:40.791212    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d4c6b64d6-sslfl" podUID="a91910cc-f372-4202-be6c-bec3e601c18e" containerName="console" containerID="cri-o://ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600" gracePeriod=15
Sep 10 21:17:41.040529 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.040494    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:17:41.176064 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176029    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-trusted-ca-bundle\") pod \"a91910cc-f372-4202-be6c-bec3e601c18e\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") "
Sep 10 21:17:41.176262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176082    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-service-ca\") pod \"a91910cc-f372-4202-be6c-bec3e601c18e\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") "
Sep 10 21:17:41.176262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176113    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-oauth-serving-cert\") pod \"a91910cc-f372-4202-be6c-bec3e601c18e\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") "
Sep 10 21:17:41.176262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176166    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-serving-cert\") pod \"a91910cc-f372-4202-be6c-bec3e601c18e\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") "
Sep 10 21:17:41.176262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176210    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-console-config\") pod \"a91910cc-f372-4202-be6c-bec3e601c18e\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") "
Sep 10 21:17:41.176262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176246    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-oauth-config\") pod \"a91910cc-f372-4202-be6c-bec3e601c18e\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") "
Sep 10 21:17:41.176534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176268    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqwzg\" (UniqueName: \"kubernetes.io/projected/a91910cc-f372-4202-be6c-bec3e601c18e-kube-api-access-qqwzg\") pod \"a91910cc-f372-4202-be6c-bec3e601c18e\" (UID: \"a91910cc-f372-4202-be6c-bec3e601c18e\") "
Sep 10 21:17:41.176534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176478    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a91910cc-f372-4202-be6c-bec3e601c18e" (UID: "a91910cc-f372-4202-be6c-bec3e601c18e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:41.176688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176663    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-service-ca" (OuterVolumeSpecName: "service-ca") pod "a91910cc-f372-4202-be6c-bec3e601c18e" (UID: "a91910cc-f372-4202-be6c-bec3e601c18e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:41.176743 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176697    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a91910cc-f372-4202-be6c-bec3e601c18e" (UID: "a91910cc-f372-4202-be6c-bec3e601c18e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:41.176809 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.176764    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-console-config" (OuterVolumeSpecName: "console-config") pod "a91910cc-f372-4202-be6c-bec3e601c18e" (UID: "a91910cc-f372-4202-be6c-bec3e601c18e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:41.178474 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.178446    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a91910cc-f372-4202-be6c-bec3e601c18e" (UID: "a91910cc-f372-4202-be6c-bec3e601c18e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:17:41.178599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.178530    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a91910cc-f372-4202-be6c-bec3e601c18e" (UID: "a91910cc-f372-4202-be6c-bec3e601c18e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:17:41.178599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.178541    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91910cc-f372-4202-be6c-bec3e601c18e-kube-api-access-qqwzg" (OuterVolumeSpecName: "kube-api-access-qqwzg") pod "a91910cc-f372-4202-be6c-bec3e601c18e" (UID: "a91910cc-f372-4202-be6c-bec3e601c18e"). InnerVolumeSpecName "kube-api-access-qqwzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:41.277599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.277574    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-console-config\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:41.277599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.277598    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-oauth-config\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:41.277791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.277612    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqwzg\" (UniqueName: \"kubernetes.io/projected/a91910cc-f372-4202-be6c-bec3e601c18e-kube-api-access-qqwzg\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:41.277791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.277624    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-trusted-ca-bundle\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:41.277791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.277638    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:41.277791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.277649    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a91910cc-f372-4202-be6c-bec3e601c18e-oauth-serving-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:41.277791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.277660    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91910cc-f372-4202-be6c-bec3e601c18e-console-serving-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:17:41.822804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.822766    2583 generic.go:356] "Generic (PLEG): container finished" podID="a91910cc-f372-4202-be6c-bec3e601c18e" containerID="ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600" exitCode=0
Sep 10 21:17:41.823196 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.822816    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4c6b64d6-sslfl"
Sep 10 21:17:41.823196 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.822834    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4c6b64d6-sslfl" event={"ID":"a91910cc-f372-4202-be6c-bec3e601c18e","Type":"ContainerDied","Data":"ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600"}
Sep 10 21:17:41.823196 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.822914    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4c6b64d6-sslfl" event={"ID":"a91910cc-f372-4202-be6c-bec3e601c18e","Type":"ContainerDied","Data":"0a8c236b61701b20b1e8e872805457e429eddde371c39a468f40c1640eef00b5"}
Sep 10 21:17:41.823196 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.822940    2583 scope.go:117] "RemoveContainer" containerID="ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600"
Sep 10 21:17:41.831247 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.831228    2583 scope.go:117] "RemoveContainer" containerID="ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600"
Sep 10 21:17:41.831556 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:17:41.831536    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600\": container with ID starting with ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600 not found: ID does not exist" containerID="ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600"
Sep 10 21:17:41.831630 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.831564    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600"} err="failed to get container status \"ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600\": rpc error: code = NotFound desc = could not find container \"ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600\": container with ID starting with ea2a7b269a955b18363914393d65978f90418957c8d6f0af202b76748ebfb600 not found: ID does not exist"
Sep 10 21:17:41.843029 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.843008    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d4c6b64d6-sslfl"]
Sep 10 21:17:41.846554 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.846533    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d4c6b64d6-sslfl"]
Sep 10 21:17:41.959678 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:17:41.959650    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a91910cc-f372-4202-be6c-bec3e601c18e" path="/var/lib/kubelet/pods/a91910cc-f372-4202-be6c-bec3e601c18e/volumes"
Sep 10 21:18:06.800316 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:06.800278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:06.898631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:06.898600    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:19.134468 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:19.134434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:21.173016 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.172978    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"]
Sep 10 21:18:21.173397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.173287    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2254a779-b871-4af2-8530-bf35f67ce8a2" containerName="istio-proxy"
Sep 10 21:18:21.173397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.173299    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2254a779-b871-4af2-8530-bf35f67ce8a2" containerName="istio-proxy"
Sep 10 21:18:21.173397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.173312    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91910cc-f372-4202-be6c-bec3e601c18e" containerName="console"
Sep 10 21:18:21.173397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.173320    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91910cc-f372-4202-be6c-bec3e601c18e" containerName="console"
Sep 10 21:18:21.173397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.173371    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2254a779-b871-4af2-8530-bf35f67ce8a2" containerName="istio-proxy"
Sep 10 21:18:21.173397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.173379    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91910cc-f372-4202-be6c-bec3e601c18e" containerName="console"
Sep 10 21:18:21.176235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.176219    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:21.177965 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.177945    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:18:21.178081 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.178028    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2qdbl\""
Sep 10 21:18:21.178123 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.178098    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:18:21.183698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.183677    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"]
Sep 10 21:18:21.194198 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.194173    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxcvv\" (UniqueName: \"kubernetes.io/projected/b33f7f7f-c0cb-47a7-8139-072ad774df1d-kube-api-access-dxcvv\") pod \"limitador-operator-controller-manager-74495c69f8-mfqfn\" (UID: \"b33f7f7f-c0cb-47a7-8139-072ad774df1d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:21.295543 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.295493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxcvv\" (UniqueName: \"kubernetes.io/projected/b33f7f7f-c0cb-47a7-8139-072ad774df1d-kube-api-access-dxcvv\") pod \"limitador-operator-controller-manager-74495c69f8-mfqfn\" (UID: \"b33f7f7f-c0cb-47a7-8139-072ad774df1d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:21.308584 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.308554    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxcvv\" (UniqueName: \"kubernetes.io/projected/b33f7f7f-c0cb-47a7-8139-072ad774df1d-kube-api-access-dxcvv\") pod \"limitador-operator-controller-manager-74495c69f8-mfqfn\" (UID: \"b33f7f7f-c0cb-47a7-8139-072ad774df1d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:21.487084 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.487001    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:21.735047 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.735010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:21.754590 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.754515    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"]
Sep 10 21:18:21.757350 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:18:21.757325    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb33f7f7f_c0cb_47a7_8139_072ad774df1d.slice/crio-d5e61928aefb54eac68688d37c962724833f1f3ac2d4b0256cc28609e957cda3 WatchSource:0}: Error finding container d5e61928aefb54eac68688d37c962724833f1f3ac2d4b0256cc28609e957cda3: Status 404 returned error can't find the container with id d5e61928aefb54eac68688d37c962724833f1f3ac2d4b0256cc28609e957cda3
Sep 10 21:18:21.989495 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:21.989459    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn" event={"ID":"b33f7f7f-c0cb-47a7-8139-072ad774df1d","Type":"ContainerStarted","Data":"d5e61928aefb54eac68688d37c962724833f1f3ac2d4b0256cc28609e957cda3"}
Sep 10 21:18:22.133680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:22.133578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:22.551211 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:22.551180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:25.005959 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:25.005919    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn" event={"ID":"b33f7f7f-c0cb-47a7-8139-072ad774df1d","Type":"ContainerStarted","Data":"ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25"}
Sep 10 21:18:25.006414 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:25.005998    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:25.030933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:25.030743    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn" podStartSLOduration=1.441100944 podStartE2EDuration="4.030725904s" totalImagesPullingTime="2.58962496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:25.030071221 +0000 UTC m=+495.557035568" watchObservedRunningTime="2026-09-10 21:18:25.030725904 +0000 UTC m=+495.557690251"
Sep 10 21:18:25.412007 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:25.411967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:25.534164 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:25.534133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:26.136611 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:26.136580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:26.421241 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:26.421209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:26.521971 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:26.521939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:28.148176 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:28.148122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:28.253843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:28.253805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:28.758481 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:28.758447    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:29.508169 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:29.508128    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:30.596244 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.596213    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"]
Sep 10 21:18:30.598547 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.598530    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"
Sep 10 21:18:30.600410 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.600386    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 21:18:30.600549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.600420    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2sqwt\""
Sep 10 21:18:30.608843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.608823    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"]
Sep 10 21:18:30.674290 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.674258    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdgn\" (UniqueName: \"kubernetes.io/projected/d6be1783-b68d-477a-8778-54832cd79db2-kube-api-access-cwdgn\") pod \"dns-operator-controller-manager-7b84cd7c9c-rbhsb\" (UID: \"d6be1783-b68d-477a-8778-54832cd79db2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"
Sep 10 21:18:30.774749 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.774715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdgn\" (UniqueName: \"kubernetes.io/projected/d6be1783-b68d-477a-8778-54832cd79db2-kube-api-access-cwdgn\") pod \"dns-operator-controller-manager-7b84cd7c9c-rbhsb\" (UID: \"d6be1783-b68d-477a-8778-54832cd79db2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"
Sep 10 21:18:30.783851 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.783828    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdgn\" (UniqueName: \"kubernetes.io/projected/d6be1783-b68d-477a-8778-54832cd79db2-kube-api-access-cwdgn\") pod \"dns-operator-controller-manager-7b84cd7c9c-rbhsb\" (UID: \"d6be1783-b68d-477a-8778-54832cd79db2\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"
Sep 10 21:18:30.908418 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:30.908391    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"
Sep 10 21:18:31.179222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:31.179148    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"]
Sep 10 21:18:31.182665 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:18:31.182636    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6be1783_b68d_477a_8778_54832cd79db2.slice/crio-6529b7e59275fec51383fbef1b0c3cc26c95e04d02e591c3db80d2eac4a94581 WatchSource:0}: Error finding container 6529b7e59275fec51383fbef1b0c3cc26c95e04d02e591c3db80d2eac4a94581: Status 404 returned error can't find the container with id 6529b7e59275fec51383fbef1b0c3cc26c95e04d02e591c3db80d2eac4a94581
Sep 10 21:18:31.812929 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:31.812892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:31.957819 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:31.957780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:32.038415 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:32.038373    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb" event={"ID":"d6be1783-b68d-477a-8778-54832cd79db2","Type":"ContainerStarted","Data":"6529b7e59275fec51383fbef1b0c3cc26c95e04d02e591c3db80d2eac4a94581"}
Sep 10 21:18:32.649314 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:32.649246    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:32.907337 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:32.907273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:33.044737 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:33.044702    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb" event={"ID":"d6be1783-b68d-477a-8778-54832cd79db2","Type":"ContainerStarted","Data":"29cd55fcb7ff11612d6035541a5ad5d29f0134aff224f6f0d41e5ff2c6756466"}
Sep 10 21:18:33.044901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:33.044836    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"
Sep 10 21:18:33.081971 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:33.081923    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb" podStartSLOduration=1.429874606 podStartE2EDuration="3.081907064s" totalImagesPullingTime="1.652032458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:33.080371977 +0000 UTC m=+503.607336348" watchObservedRunningTime="2026-09-10 21:18:33.081907064 +0000 UTC m=+503.608871410"
Sep 10 21:18:35.730127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:35.730099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:35.787096 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:35.787060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:36.013709 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:36.013629    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:36.134981 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:36.134949    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:36.233810 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:36.233779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:36.336899 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:36.336828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:36.460897 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:36.460868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:43.987301 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:43.987268    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"]
Sep 10 21:18:43.987786 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:43.987523    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" containerName="manager" containerID="cri-o://ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25" gracePeriod=2
Sep 10 21:18:44.001617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.001589    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"]
Sep 10 21:18:44.009144 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:44.009104    2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b33f7f7f-c0cb-47a7-8139-072ad774df1d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-mfqfn\": pods \"limitador-operator-controller-manager-74495c69f8-mfqfn\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:44.019737 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.019714    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"]
Sep 10 21:18:44.019998 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.019987    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" containerName="manager"
Sep 10 21:18:44.020057 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.020000    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" containerName="manager"
Sep 10 21:18:44.020108 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.020060    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" containerName="manager"
Sep 10 21:18:44.022024 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.022005    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"
Sep 10 21:18:44.037355 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.037329    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"]
Sep 10 21:18:44.052415 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:44.052374    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mfqfn\" is forbidden: User \"system:node:ip-10-0-141-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-146.ec2.internal' and this object" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:44.055981 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.055955    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-rbhsb"
Sep 10 21:18:44.057631 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:44.057602    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mfqfn\" is forbidden: User \"system:node:ip-10-0-141-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-146.ec2.internal' and this object" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:44.079717 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.079681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t459\" (UniqueName: \"kubernetes.io/projected/bd0ad69e-5685-4d6f-a385-876e84cd6a25-kube-api-access-8t459\") pod \"limitador-operator-controller-manager-74495c69f8-52pbp\" (UID: \"bd0ad69e-5685-4d6f-a385-876e84cd6a25\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"
Sep 10 21:18:44.180276 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.180246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t459\" (UniqueName: \"kubernetes.io/projected/bd0ad69e-5685-4d6f-a385-876e84cd6a25-kube-api-access-8t459\") pod \"limitador-operator-controller-manager-74495c69f8-52pbp\" (UID: \"bd0ad69e-5685-4d6f-a385-876e84cd6a25\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"
Sep 10 21:18:44.189299 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.189270    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t459\" (UniqueName: \"kubernetes.io/projected/bd0ad69e-5685-4d6f-a385-876e84cd6a25-kube-api-access-8t459\") pod \"limitador-operator-controller-manager-74495c69f8-52pbp\" (UID: \"bd0ad69e-5685-4d6f-a385-876e84cd6a25\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"
Sep 10 21:18:44.222204 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.222182    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:44.223640 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:44.223617    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mfqfn\" is forbidden: User \"system:node:ip-10-0-141-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-146.ec2.internal' and this object" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:44.279092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.279015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:44.280989 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.280967    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxcvv\" (UniqueName: \"kubernetes.io/projected/b33f7f7f-c0cb-47a7-8139-072ad774df1d-kube-api-access-dxcvv\") pod \"b33f7f7f-c0cb-47a7-8139-072ad774df1d\" (UID: \"b33f7f7f-c0cb-47a7-8139-072ad774df1d\") "
Sep 10 21:18:44.283141 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.283121    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b33f7f7f-c0cb-47a7-8139-072ad774df1d-kube-api-access-dxcvv" (OuterVolumeSpecName: "kube-api-access-dxcvv") pod "b33f7f7f-c0cb-47a7-8139-072ad774df1d" (UID: "b33f7f7f-c0cb-47a7-8139-072ad774df1d"). InnerVolumeSpecName "kube-api-access-dxcvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:44.361902 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.361872    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"
Sep 10 21:18:44.381954 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.381913    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxcvv\" (UniqueName: \"kubernetes.io/projected/b33f7f7f-c0cb-47a7-8139-072ad774df1d-kube-api-access-dxcvv\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:18:44.382187 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.382164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:44.518894 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.518861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:44.669846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:44.669816    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"]
Sep 10 21:18:44.673575 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:18:44.673547    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0ad69e_5685_4d6f_a385_876e84cd6a25.slice/crio-39a4ffa2d852404bb30062e45ec44c2ea8b9278890ad7f01011f6252418a964b WatchSource:0}: Error finding container 39a4ffa2d852404bb30062e45ec44c2ea8b9278890ad7f01011f6252418a964b: Status 404 returned error can't find the container with id 39a4ffa2d852404bb30062e45ec44c2ea8b9278890ad7f01011f6252418a964b
Sep 10 21:18:45.104845 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.104760    2583 generic.go:356] "Generic (PLEG): container finished" podID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" containerID="ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25" exitCode=0
Sep 10 21:18:45.104845 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.104802    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:45.105300 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.104863    2583 scope.go:117] "RemoveContainer" containerID="ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25"
Sep 10 21:18:45.106704 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:45.106673    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mfqfn\" is forbidden: User \"system:node:ip-10-0-141-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-146.ec2.internal' and this object" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:45.106801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.106774    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp" event={"ID":"bd0ad69e-5685-4d6f-a385-876e84cd6a25","Type":"ContainerStarted","Data":"d62f9aa5aa1979ecf690fee8fe4a2073b0495d3c12c68128af1da80abad209f0"}
Sep 10 21:18:45.106858 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.106820    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp" event={"ID":"bd0ad69e-5685-4d6f-a385-876e84cd6a25","Type":"ContainerStarted","Data":"39a4ffa2d852404bb30062e45ec44c2ea8b9278890ad7f01011f6252418a964b"}
Sep 10 21:18:45.106902 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.106882    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"
Sep 10 21:18:45.113317 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.113294    2583 scope.go:117] "RemoveContainer" containerID="ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25"
Sep 10 21:18:45.113585 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:45.113564    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25\": container with ID starting with ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25 not found: ID does not exist" containerID="ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25"
Sep 10 21:18:45.113673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.113592    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25"} err="failed to get container status \"ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25\": rpc error: code = NotFound desc = could not find container \"ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25\": container with ID starting with ade315da4c01f28f95d99b1be8a9703da52097c65a3ea33ef81f7f9a6809aa25 not found: ID does not exist"
Sep 10 21:18:45.123711 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.123668    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp" podStartSLOduration=2.123656347 podStartE2EDuration="2.123656347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:45.12258 +0000 UTC m=+515.649544346" watchObservedRunningTime="2026-09-10 21:18:45.123656347 +0000 UTC m=+515.650620694"
Sep 10 21:18:45.124290 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:45.124268    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mfqfn\" is forbidden: User \"system:node:ip-10-0-141-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-146.ec2.internal' and this object" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:45.125340 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:18:45.125317    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mfqfn\" is forbidden: User \"system:node:ip-10-0-141-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-146.ec2.internal' and this object" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mfqfn"
Sep 10 21:18:45.960293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:45.960253    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b33f7f7f-c0cb-47a7-8139-072ad774df1d" path="/var/lib/kubelet/pods/b33f7f7f-c0cb-47a7-8139-072ad774df1d/volumes"
Sep 10 21:18:46.747386 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:46.747356    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:46.843489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:46.843435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:51.063605 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:51.063565    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:51.164064 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:51.164027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:54.899146 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:54.899111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:54.999175 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:54.999144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:55.098278 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:55.098219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:55.163787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:55.163693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:56.094896 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:56.094861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:56.114431 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:56.114407    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-52pbp"
Sep 10 21:18:56.188906 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:18:56.188869    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:05.299732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:05.299645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:05.397766 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:05.397732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:07.735868 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:07.735835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:07.793029 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:07.792994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:12.442214 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.442160    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"]
Sep 10 21:19:12.445427 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.445408    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.447389 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.447371    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lwgzz\""
Sep 10 21:19:12.463916 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.463892    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"]
Sep 10 21:19:12.503350 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.503317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:12.621308 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621270    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621316    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621343    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621441    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621482    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad5f6765-9edd-47fc-a141-d780d3aff1af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkpgh\" (UniqueName: \"kubernetes.io/projected/ad5f6765-9edd-47fc-a141-d780d3aff1af-kube-api-access-dkpgh\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.621701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.621636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.673253 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.673222    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:12.722525 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722525 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722690 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722596    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722690 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722626    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722755    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722819    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722949 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722949 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722901    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.722949 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722905    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad5f6765-9edd-47fc-a141-d780d3aff1af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.723105 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.722963    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkpgh\" (UniqueName: \"kubernetes.io/projected/ad5f6765-9edd-47fc-a141-d780d3aff1af-kube-api-access-dkpgh\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.723105 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.723061    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.723245 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.723227    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.723566 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.723539    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad5f6765-9edd-47fc-a141-d780d3aff1af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.724950 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.724930    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.725318 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.725297    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.731432 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.731403    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkpgh\" (UniqueName: \"kubernetes.io/projected/ad5f6765-9edd-47fc-a141-d780d3aff1af-kube-api-access-dkpgh\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.733069 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.733046    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad5f6765-9edd-47fc-a141-d780d3aff1af-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dc9wf\" (UID: \"ad5f6765-9edd-47fc-a141-d780d3aff1af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:12.756074 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:12.756047    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:13.031192 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.030977    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"]
Sep 10 21:19:13.033852 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:19:13.033826    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad5f6765_9edd_47fc_a141_d780d3aff1af.slice/crio-ec15bd611ae0c274d2ea5ed33696cf803210d5b3a803df148679d4c2162fc71b WatchSource:0}: Error finding container ec15bd611ae0c274d2ea5ed33696cf803210d5b3a803df148679d4c2162fc71b: Status 404 returned error can't find the container with id ec15bd611ae0c274d2ea5ed33696cf803210d5b3a803df148679d4c2162fc71b
Sep 10 21:19:13.036059 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.036029    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:19:13.036132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.036091    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:19:13.036132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.036123    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:19:13.224074 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.224042    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" event={"ID":"ad5f6765-9edd-47fc-a141-d780d3aff1af","Type":"ContainerStarted","Data":"f2d596b0f7eda003b873416372e7b1711b574083e9bf4157c82ffc1317acb8c8"}
Sep 10 21:19:13.224296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.224096    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" event={"ID":"ad5f6765-9edd-47fc-a141-d780d3aff1af","Type":"ContainerStarted","Data":"ec15bd611ae0c274d2ea5ed33696cf803210d5b3a803df148679d4c2162fc71b"}
Sep 10 21:19:13.244271 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.244224    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" podStartSLOduration=1.244209044 podStartE2EDuration="1.244209044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:13.24266378 +0000 UTC m=+543.769628127" watchObservedRunningTime="2026-09-10 21:19:13.244209044 +0000 UTC m=+543.771173392"
Sep 10 21:19:13.756175 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:13.756118    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:14.761571 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:14.761543    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:15.232137 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:15.232107    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:15.233030 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:15.233013    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:19:26.180123 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.180086    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jbjj"]
Sep 10 21:19:26.183152 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.183128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.185435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.185411    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5x6vr\""
Sep 10 21:19:26.185569 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.185447    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 21:19:26.188319 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.188293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:26.194334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.194312    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jbjj"]
Sep 10 21:19:26.228775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.228738    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4mh2\" (UniqueName: \"kubernetes.io/projected/3f2c202e-66e4-47a9-ab49-8426685e1ca4-kube-api-access-b4mh2\") pod \"limitador-limitador-8588749977-6jbjj\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") " pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.228910 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.228834    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f2c202e-66e4-47a9-ab49-8426685e1ca4-config-file\") pod \"limitador-limitador-8588749977-6jbjj\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") " pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.281970 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.281937    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jbjj"]
Sep 10 21:19:26.290329 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.290301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:26.329522 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.329477    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4mh2\" (UniqueName: \"kubernetes.io/projected/3f2c202e-66e4-47a9-ab49-8426685e1ca4-kube-api-access-b4mh2\") pod \"limitador-limitador-8588749977-6jbjj\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") " pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.329646 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.329565    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f2c202e-66e4-47a9-ab49-8426685e1ca4-config-file\") pod \"limitador-limitador-8588749977-6jbjj\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") " pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.330205 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.330182    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f2c202e-66e4-47a9-ab49-8426685e1ca4-config-file\") pod \"limitador-limitador-8588749977-6jbjj\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") " pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.337080 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.337056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4mh2\" (UniqueName: \"kubernetes.io/projected/3f2c202e-66e4-47a9-ab49-8426685e1ca4-kube-api-access-b4mh2\") pod \"limitador-limitador-8588749977-6jbjj\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") " pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.349407 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.349382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:26.494599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.494522    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:26.766631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:26.766399    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jbjj"]
Sep 10 21:19:26.769330 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:19:26.769302    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f2c202e_66e4_47a9_ab49_8426685e1ca4.slice/crio-e2513e336558cd03a1cc83400f49a5b7a1ab87ebfbd749a0b8e497d63f85b341 WatchSource:0}: Error finding container e2513e336558cd03a1cc83400f49a5b7a1ab87ebfbd749a0b8e497d63f85b341: Status 404 returned error can't find the container with id e2513e336558cd03a1cc83400f49a5b7a1ab87ebfbd749a0b8e497d63f85b341
Sep 10 21:19:27.117656 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.117570    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-vhz5t"]
Sep 10 21:19:27.120699 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.120681    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-vhz5t"
Sep 10 21:19:27.122822 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.122801    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d5rjx\""
Sep 10 21:19:27.126802 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.126780    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-vhz5t"]
Sep 10 21:19:27.135364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.135341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnff9\" (UniqueName: \"kubernetes.io/projected/13ae7366-10ab-4ef1-ace9-35a0dced9959-kube-api-access-hnff9\") pod \"authorino-59c974bd48-vhz5t\" (UID: \"13ae7366-10ab-4ef1-ace9-35a0dced9959\") " pod="kuadrant-system/authorino-59c974bd48-vhz5t"
Sep 10 21:19:27.152601 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.152572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:27.220624 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.220574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:27.236701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.236675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnff9\" (UniqueName: \"kubernetes.io/projected/13ae7366-10ab-4ef1-ace9-35a0dced9959-kube-api-access-hnff9\") pod \"authorino-59c974bd48-vhz5t\" (UID: \"13ae7366-10ab-4ef1-ace9-35a0dced9959\") " pod="kuadrant-system/authorino-59c974bd48-vhz5t"
Sep 10 21:19:27.244000 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.243979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnff9\" (UniqueName: \"kubernetes.io/projected/13ae7366-10ab-4ef1-ace9-35a0dced9959-kube-api-access-hnff9\") pod \"authorino-59c974bd48-vhz5t\" (UID: \"13ae7366-10ab-4ef1-ace9-35a0dced9959\") " pod="kuadrant-system/authorino-59c974bd48-vhz5t"
Sep 10 21:19:27.283493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.283457    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj" event={"ID":"3f2c202e-66e4-47a9-ab49-8426685e1ca4","Type":"ContainerStarted","Data":"e2513e336558cd03a1cc83400f49a5b7a1ab87ebfbd749a0b8e497d63f85b341"}
Sep 10 21:19:27.430588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.430558    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-vhz5t"
Sep 10 21:19:27.705868 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:27.705422    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-vhz5t"]
Sep 10 21:19:27.742311 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:19:27.742278    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13ae7366_10ab_4ef1_ace9_35a0dced9959.slice/crio-eca18e458e247c134157310a34155f813e206678989713aa2df400f870a68cba WatchSource:0}: Error finding container eca18e458e247c134157310a34155f813e206678989713aa2df400f870a68cba: Status 404 returned error can't find the container with id eca18e458e247c134157310a34155f813e206678989713aa2df400f870a68cba
Sep 10 21:19:28.290012 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:28.289978    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-vhz5t" event={"ID":"13ae7366-10ab-4ef1-ace9-35a0dced9959","Type":"ContainerStarted","Data":"eca18e458e247c134157310a34155f813e206678989713aa2df400f870a68cba"}
Sep 10 21:19:28.291349 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:28.291328    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj" event={"ID":"3f2c202e-66e4-47a9-ab49-8426685e1ca4","Type":"ContainerStarted","Data":"506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa"}
Sep 10 21:19:28.291485 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:28.291469    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:28.307043 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:28.306996    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj" podStartSLOduration=1.306148453 podStartE2EDuration="2.306980582s" totalImagesPullingTime="1.000832129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:28.306373815 +0000 UTC m=+558.833338159" watchObservedRunningTime="2026-09-10 21:19:28.306980582 +0000 UTC m=+558.833944929"
Sep 10 21:19:30.303034 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:30.302995    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-vhz5t" event={"ID":"13ae7366-10ab-4ef1-ace9-35a0dced9959","Type":"ContainerStarted","Data":"947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08"}
Sep 10 21:19:30.320042 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:30.319993    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-vhz5t" podStartSLOduration=1.6876385900000002 podStartE2EDuration="3.319978381s" totalImagesPullingTime="1.632339791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:30.318584028 +0000 UTC m=+560.845548370" watchObservedRunningTime="2026-09-10 21:19:30.319978381 +0000 UTC m=+560.846942727"
Sep 10 21:19:39.298903 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:39.298876    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:44.261979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.261944    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jbjj"]
Sep 10 21:19:44.262396 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.262150    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj" podUID="3f2c202e-66e4-47a9-ab49-8426685e1ca4" containerName="limitador" containerID="cri-o://506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa" gracePeriod=30
Sep 10 21:19:44.803180 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.803158    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:44.882920 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.882835    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4mh2\" (UniqueName: \"kubernetes.io/projected/3f2c202e-66e4-47a9-ab49-8426685e1ca4-kube-api-access-b4mh2\") pod \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") "
Sep 10 21:19:44.882920 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.882883    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f2c202e-66e4-47a9-ab49-8426685e1ca4-config-file\") pod \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\" (UID: \"3f2c202e-66e4-47a9-ab49-8426685e1ca4\") "
Sep 10 21:19:44.883262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.883235    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f2c202e-66e4-47a9-ab49-8426685e1ca4-config-file" (OuterVolumeSpecName: "config-file") pod "3f2c202e-66e4-47a9-ab49-8426685e1ca4" (UID: "3f2c202e-66e4-47a9-ab49-8426685e1ca4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:19:44.885011 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.884975    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f2c202e-66e4-47a9-ab49-8426685e1ca4-kube-api-access-b4mh2" (OuterVolumeSpecName: "kube-api-access-b4mh2") pod "3f2c202e-66e4-47a9-ab49-8426685e1ca4" (UID: "3f2c202e-66e4-47a9-ab49-8426685e1ca4"). InnerVolumeSpecName "kube-api-access-b4mh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:19:44.984572 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.984543    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4mh2\" (UniqueName: \"kubernetes.io/projected/3f2c202e-66e4-47a9-ab49-8426685e1ca4-kube-api-access-b4mh2\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:19:44.984572 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:44.984570    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f2c202e-66e4-47a9-ab49-8426685e1ca4-config-file\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:19:45.370173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.370137    2583 generic.go:356] "Generic (PLEG): container finished" podID="3f2c202e-66e4-47a9-ab49-8426685e1ca4" containerID="506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa" exitCode=0
Sep 10 21:19:45.370173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.370176    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj"
Sep 10 21:19:45.370695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.370200    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj" event={"ID":"3f2c202e-66e4-47a9-ab49-8426685e1ca4","Type":"ContainerDied","Data":"506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa"}
Sep 10 21:19:45.370695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.370245    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6jbjj" event={"ID":"3f2c202e-66e4-47a9-ab49-8426685e1ca4","Type":"ContainerDied","Data":"e2513e336558cd03a1cc83400f49a5b7a1ab87ebfbd749a0b8e497d63f85b341"}
Sep 10 21:19:45.370695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.370267    2583 scope.go:117] "RemoveContainer" containerID="506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa"
Sep 10 21:19:45.379847 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.379830    2583 scope.go:117] "RemoveContainer" containerID="506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa"
Sep 10 21:19:45.380082 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:19:45.380062    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa\": container with ID starting with 506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa not found: ID does not exist" containerID="506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa"
Sep 10 21:19:45.380198 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.380175    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa"} err="failed to get container status \"506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa\": rpc error: code = NotFound desc = could not find container \"506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa\": container with ID starting with 506fabcec3e05be633901d95e99d95fc6fc316658f9d4c62a57c37cec1a300aa not found: ID does not exist"
Sep 10 21:19:45.390632 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.390598    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jbjj"]
Sep 10 21:19:45.392273 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.392247    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6jbjj"]
Sep 10 21:19:45.631902 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.631797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:45.726872 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.726842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:45.960730 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:45.960699    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f2c202e-66e4-47a9-ab49-8426685e1ca4" path="/var/lib/kubelet/pods/3f2c202e-66e4-47a9-ab49-8426685e1ca4/volumes"
Sep 10 21:19:46.082052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.082013    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"]
Sep 10 21:19:46.082479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.082459    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f2c202e-66e4-47a9-ab49-8426685e1ca4" containerName="limitador"
Sep 10 21:19:46.082479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.082479    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f2c202e-66e4-47a9-ab49-8426685e1ca4" containerName="limitador"
Sep 10 21:19:46.082642 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.082572    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f2c202e-66e4-47a9-ab49-8426685e1ca4" containerName="limitador"
Sep 10 21:19:46.085603 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.085584    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.087817 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.087796    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 21:19:46.087909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.087827    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:19:46.088262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.088241    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:19:46.088363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.088266    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fr5kk\""
Sep 10 21:19:46.093941 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.093921    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"]
Sep 10 21:19:46.107219 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.107196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:46.193681 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.193646    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2v8q\" (UniqueName: \"kubernetes.io/projected/f723b6f4-e96f-45f8-8b0f-251f6483926d-kube-api-access-l2v8q\") pod \"postgres-69fb4594cf-p2fdf\" (UID: \"f723b6f4-e96f-45f8-8b0f-251f6483926d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.193842 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.193689    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f723b6f4-e96f-45f8-8b0f-251f6483926d-data\") pod \"postgres-69fb4594cf-p2fdf\" (UID: \"f723b6f4-e96f-45f8-8b0f-251f6483926d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.294733 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.294645    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2v8q\" (UniqueName: \"kubernetes.io/projected/f723b6f4-e96f-45f8-8b0f-251f6483926d-kube-api-access-l2v8q\") pod \"postgres-69fb4594cf-p2fdf\" (UID: \"f723b6f4-e96f-45f8-8b0f-251f6483926d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.294733 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.294687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f723b6f4-e96f-45f8-8b0f-251f6483926d-data\") pod \"postgres-69fb4594cf-p2fdf\" (UID: \"f723b6f4-e96f-45f8-8b0f-251f6483926d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.295111 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.295092    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f723b6f4-e96f-45f8-8b0f-251f6483926d-data\") pod \"postgres-69fb4594cf-p2fdf\" (UID: \"f723b6f4-e96f-45f8-8b0f-251f6483926d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.302713 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.302687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2v8q\" (UniqueName: \"kubernetes.io/projected/f723b6f4-e96f-45f8-8b0f-251f6483926d-kube-api-access-l2v8q\") pod \"postgres-69fb4594cf-p2fdf\" (UID: \"f723b6f4-e96f-45f8-8b0f-251f6483926d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.396238 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.396216    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:46.666157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:46.666133    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"]
Sep 10 21:19:46.668557 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:19:46.668526    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf723b6f4_e96f_45f8_8b0f_251f6483926d.slice/crio-386ce2e24e03168ea22fd498fb850e4c7425971ede48cfc080a6887e9a8ce874 WatchSource:0}: Error finding container 386ce2e24e03168ea22fd498fb850e4c7425971ede48cfc080a6887e9a8ce874: Status 404 returned error can't find the container with id 386ce2e24e03168ea22fd498fb850e4c7425971ede48cfc080a6887e9a8ce874
Sep 10 21:19:47.381443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:47.381364    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf" event={"ID":"f723b6f4-e96f-45f8-8b0f-251f6483926d","Type":"ContainerStarted","Data":"386ce2e24e03168ea22fd498fb850e4c7425971ede48cfc080a6887e9a8ce874"}
Sep 10 21:19:52.407474 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:52.407437    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf" event={"ID":"f723b6f4-e96f-45f8-8b0f-251f6483926d","Type":"ContainerStarted","Data":"c52dc1c4692528271f6439d153dcd950447841ea7bbe648665f6f1e401950656"}
Sep 10 21:19:52.408005 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:52.407566    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:52.424338 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:52.424278    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf" podStartSLOduration=1.246202565 podStartE2EDuration="6.424255808s" totalImagesPullingTime="5.178053243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:52.423248814 +0000 UTC m=+582.950213161" watchObservedRunningTime="2026-09-10 21:19:52.424255808 +0000 UTC m=+582.951220157"
Sep 10 21:19:58.440705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:58.440672    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p2fdf"
Sep 10 21:19:58.743366 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:58.743275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:59.224922 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.224891    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-86489fd7bc-9fsrz"]
Sep 10 21:19:59.227840 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.227824    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86489fd7bc-9fsrz"
Sep 10 21:19:59.236520 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.236477    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86489fd7bc-9fsrz"]
Sep 10 21:19:59.315988 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.315957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v29sw\" (UniqueName: \"kubernetes.io/projected/c4d4c35c-fe77-4585-b60a-493d58f877d2-kube-api-access-v29sw\") pod \"authorino-86489fd7bc-9fsrz\" (UID: \"c4d4c35c-fe77-4585-b60a-493d58f877d2\") " pod="kuadrant-system/authorino-86489fd7bc-9fsrz"
Sep 10 21:19:59.417520 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.417390    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v29sw\" (UniqueName: \"kubernetes.io/projected/c4d4c35c-fe77-4585-b60a-493d58f877d2-kube-api-access-v29sw\") pod \"authorino-86489fd7bc-9fsrz\" (UID: \"c4d4c35c-fe77-4585-b60a-493d58f877d2\") " pod="kuadrant-system/authorino-86489fd7bc-9fsrz"
Sep 10 21:19:59.426122 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.426098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v29sw\" (UniqueName: \"kubernetes.io/projected/c4d4c35c-fe77-4585-b60a-493d58f877d2-kube-api-access-v29sw\") pod \"authorino-86489fd7bc-9fsrz\" (UID: \"c4d4c35c-fe77-4585-b60a-493d58f877d2\") " pod="kuadrant-system/authorino-86489fd7bc-9fsrz"
Sep 10 21:19:59.479799 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.479720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:59.537661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.537633    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86489fd7bc-9fsrz"
Sep 10 21:19:59.827899 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:19:59.827875    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-86489fd7bc-9fsrz"]
Sep 10 21:19:59.830418 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:19:59.830389    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4d4c35c_fe77_4585_b60a_493d58f877d2.slice/crio-b3ed90b6ea2396a1a38a71ad5be70127786900c91ee058b28fdc11ae73aec572 WatchSource:0}: Error finding container b3ed90b6ea2396a1a38a71ad5be70127786900c91ee058b28fdc11ae73aec572: Status 404 returned error can't find the container with id b3ed90b6ea2396a1a38a71ad5be70127786900c91ee058b28fdc11ae73aec572
Sep 10 21:20:00.443192 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:00.443151    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86489fd7bc-9fsrz" event={"ID":"c4d4c35c-fe77-4585-b60a-493d58f877d2","Type":"ContainerStarted","Data":"b3ed90b6ea2396a1a38a71ad5be70127786900c91ee058b28fdc11ae73aec572"}
Sep 10 21:20:01.449445 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.449404    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86489fd7bc-9fsrz" event={"ID":"c4d4c35c-fe77-4585-b60a-493d58f877d2","Type":"ContainerStarted","Data":"9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8"}
Sep 10 21:20:01.465262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.465205    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-86489fd7bc-9fsrz" podStartSLOduration=1.552658458 podStartE2EDuration="2.4651878s" totalImagesPullingTime="912.529342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:01.464534857 +0000 UTC m=+591.991499199" watchObservedRunningTime="2026-09-10 21:20:01.4651878 +0000 UTC m=+591.992152146"
Sep 10 21:20:01.491653 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.491617    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-vhz5t"]
Sep 10 21:20:01.491827 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.491792    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-vhz5t" podUID="13ae7366-10ab-4ef1-ace9-35a0dced9959" containerName="authorino" containerID="cri-o://947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08" gracePeriod=30
Sep 10 21:20:01.762646 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.762618    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-vhz5t"
Sep 10 21:20:01.814683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.814648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:01.840346 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.840317    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnff9\" (UniqueName: \"kubernetes.io/projected/13ae7366-10ab-4ef1-ace9-35a0dced9959-kube-api-access-hnff9\") pod \"13ae7366-10ab-4ef1-ace9-35a0dced9959\" (UID: \"13ae7366-10ab-4ef1-ace9-35a0dced9959\") "
Sep 10 21:20:01.842366 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.842329    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ae7366-10ab-4ef1-ace9-35a0dced9959-kube-api-access-hnff9" (OuterVolumeSpecName: "kube-api-access-hnff9") pod "13ae7366-10ab-4ef1-ace9-35a0dced9959" (UID: "13ae7366-10ab-4ef1-ace9-35a0dced9959"). InnerVolumeSpecName "kube-api-access-hnff9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:01.915023 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.914988    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:01.941824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:01.941788    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnff9\" (UniqueName: \"kubernetes.io/projected/13ae7366-10ab-4ef1-ace9-35a0dced9959-kube-api-access-hnff9\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:20:02.454799 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.454762    2583 generic.go:356] "Generic (PLEG): container finished" podID="13ae7366-10ab-4ef1-ace9-35a0dced9959" containerID="947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08" exitCode=0
Sep 10 21:20:02.454799 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.454803    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-vhz5t"
Sep 10 21:20:02.455339 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.454850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-vhz5t" event={"ID":"13ae7366-10ab-4ef1-ace9-35a0dced9959","Type":"ContainerDied","Data":"947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08"}
Sep 10 21:20:02.455339 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.454902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-vhz5t" event={"ID":"13ae7366-10ab-4ef1-ace9-35a0dced9959","Type":"ContainerDied","Data":"eca18e458e247c134157310a34155f813e206678989713aa2df400f870a68cba"}
Sep 10 21:20:02.455339 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.454927    2583 scope.go:117] "RemoveContainer" containerID="947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08"
Sep 10 21:20:02.463051 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.463028    2583 scope.go:117] "RemoveContainer" containerID="947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08"
Sep 10 21:20:02.463324 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:20:02.463306    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08\": container with ID starting with 947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08 not found: ID does not exist" containerID="947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08"
Sep 10 21:20:02.463388 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.463333    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08"} err="failed to get container status \"947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08\": rpc error: code = NotFound desc = could not find container \"947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08\": container with ID starting with 947d3689c42aa39429e093849befccf628a966f95ad87363627d4780d2cc1c08 not found: ID does not exist"
Sep 10 21:20:02.474088 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.474059    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-vhz5t"]
Sep 10 21:20:02.480252 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:02.480230    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-vhz5t"]
Sep 10 21:20:03.417827 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:03.417795    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86489fd7bc-9fsrz"]
Sep 10 21:20:03.460293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:03.460258    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-86489fd7bc-9fsrz" podUID="c4d4c35c-fe77-4585-b60a-493d58f877d2" containerName="authorino" containerID="cri-o://9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8" gracePeriod=30
Sep 10 21:20:03.700613 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:03.700591    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86489fd7bc-9fsrz"
Sep 10 21:20:03.858336 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:03.858303    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v29sw\" (UniqueName: \"kubernetes.io/projected/c4d4c35c-fe77-4585-b60a-493d58f877d2-kube-api-access-v29sw\") pod \"c4d4c35c-fe77-4585-b60a-493d58f877d2\" (UID: \"c4d4c35c-fe77-4585-b60a-493d58f877d2\") "
Sep 10 21:20:03.860354 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:03.860331    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d4c35c-fe77-4585-b60a-493d58f877d2-kube-api-access-v29sw" (OuterVolumeSpecName: "kube-api-access-v29sw") pod "c4d4c35c-fe77-4585-b60a-493d58f877d2" (UID: "c4d4c35c-fe77-4585-b60a-493d58f877d2"). InnerVolumeSpecName "kube-api-access-v29sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:03.961370 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:03.959801    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v29sw\" (UniqueName: \"kubernetes.io/projected/c4d4c35c-fe77-4585-b60a-493d58f877d2-kube-api-access-v29sw\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:20:03.965689 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:03.965657    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ae7366-10ab-4ef1-ace9-35a0dced9959" path="/var/lib/kubelet/pods/13ae7366-10ab-4ef1-ace9-35a0dced9959/volumes"
Sep 10 21:20:04.466931 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.466895    2583 generic.go:356] "Generic (PLEG): container finished" podID="c4d4c35c-fe77-4585-b60a-493d58f877d2" containerID="9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8" exitCode=0
Sep 10 21:20:04.467397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.466948    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-86489fd7bc-9fsrz"
Sep 10 21:20:04.467397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.466985    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86489fd7bc-9fsrz" event={"ID":"c4d4c35c-fe77-4585-b60a-493d58f877d2","Type":"ContainerDied","Data":"9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8"}
Sep 10 21:20:04.467397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.467034    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-86489fd7bc-9fsrz" event={"ID":"c4d4c35c-fe77-4585-b60a-493d58f877d2","Type":"ContainerDied","Data":"b3ed90b6ea2396a1a38a71ad5be70127786900c91ee058b28fdc11ae73aec572"}
Sep 10 21:20:04.467397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.467057    2583 scope.go:117] "RemoveContainer" containerID="9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8"
Sep 10 21:20:04.475201 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.475177    2583 scope.go:117] "RemoveContainer" containerID="9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8"
Sep 10 21:20:04.475450 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:20:04.475430    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8\": container with ID starting with 9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8 not found: ID does not exist" containerID="9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8"
Sep 10 21:20:04.475534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.475461    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8"} err="failed to get container status \"9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8\": rpc error: code = NotFound desc = could not find container \"9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8\": container with ID starting with 9bf0c67237d72e37b1351a9522846d5c0510a93164a83d4e4a01eb9eac75b0e8 not found: ID does not exist"
Sep 10 21:20:04.481376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.481353    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-86489fd7bc-9fsrz"]
Sep 10 21:20:04.485045 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:04.485025    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-86489fd7bc-9fsrz"]
Sep 10 21:20:05.960429 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:05.960399    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d4c35c-fe77-4585-b60a-493d58f877d2" path="/var/lib/kubelet/pods/c4d4c35c-fe77-4585-b60a-493d58f877d2/volumes"
Sep 10 21:20:06.186005 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.185962    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.218231 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.218148    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65cdb78488-p6hcg"]
Sep 10 21:20:06.218645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.218618    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d4c35c-fe77-4585-b60a-493d58f877d2" containerName="authorino"
Sep 10 21:20:06.218645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.218639    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d4c35c-fe77-4585-b60a-493d58f877d2" containerName="authorino"
Sep 10 21:20:06.218817 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.218653    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ae7366-10ab-4ef1-ace9-35a0dced9959" containerName="authorino"
Sep 10 21:20:06.218817 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.218662    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ae7366-10ab-4ef1-ace9-35a0dced9959" containerName="authorino"
Sep 10 21:20:06.218817 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.218786    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ae7366-10ab-4ef1-ace9-35a0dced9959" containerName="authorino"
Sep 10 21:20:06.218817 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.218799    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d4c35c-fe77-4585-b60a-493d58f877d2" containerName="authorino"
Sep 10 21:20:06.221612 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.221589    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.223563 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.223541    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 21:20:06.223666 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.223563    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mxbbt\""
Sep 10 21:20:06.223666 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.223575    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 21:20:06.224325 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.224306    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 21:20:06.232773 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.232749    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65cdb78488-p6hcg"]
Sep 10 21:20:06.281350 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.281315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.339585 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.339552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.379278 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.379248    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-metrics-certs\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.379446 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.379300    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-webhook-cert\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.379446 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.379377    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzvwk\" (UniqueName: \"kubernetes.io/projected/5220c899-26dd-4211-86d6-53d3bff44ae1-kube-api-access-bzvwk\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.404001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.403970    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77cc466dd6-d9djh"]
Sep 10 21:20:06.406474 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.406459    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.417293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.417263    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-d9djh"]
Sep 10 21:20:06.458126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.458096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.480201 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.480122    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-metrics-certs\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.480201 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.480171    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-webhook-cert\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.480493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.480203    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzvwk\" (UniqueName: \"kubernetes.io/projected/5220c899-26dd-4211-86d6-53d3bff44ae1-kube-api-access-bzvwk\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.482580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.482560    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-metrics-certs\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.482668 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.482592    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-webhook-cert\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.488542 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.488522    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzvwk\" (UniqueName: \"kubernetes.io/projected/5220c899-26dd-4211-86d6-53d3bff44ae1-kube-api-access-bzvwk\") pod \"maas-controller-65cdb78488-p6hcg\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") " pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.534516 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.534477    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:06.580961 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.580927    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnv4p\" (UniqueName: \"kubernetes.io/projected/1bc46f35-6c0b-4406-9578-b94c0a4816c3-kube-api-access-hnv4p\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.581119 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.581003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-webhook-cert\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.581119 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.581035    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-metrics-certs\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.681970 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.681928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-webhook-cert\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.682142 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.681992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-metrics-certs\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.682142 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.682037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnv4p\" (UniqueName: \"kubernetes.io/projected/1bc46f35-6c0b-4406-9578-b94c0a4816c3-kube-api-access-hnv4p\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.684402 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.684377    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-webhook-cert\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.684402 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.684398    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-metrics-certs\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.690431 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.690407    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnv4p\" (UniqueName: \"kubernetes.io/projected/1bc46f35-6c0b-4406-9578-b94c0a4816c3-kube-api-access-hnv4p\") pod \"maas-controller-77cc466dd6-d9djh\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") " pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.719454 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.719425    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:06.810364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.810334    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65cdb78488-p6hcg"]
Sep 10 21:20:06.812128 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:20:06.812100    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5220c899_26dd_4211_86d6_53d3bff44ae1.slice/crio-d81a6edb0aca0287aef3104cdc3304d9aac7cd53792fd69cf75db9b4c06cc254 WatchSource:0}: Error finding container d81a6edb0aca0287aef3104cdc3304d9aac7cd53792fd69cf75db9b4c06cc254: Status 404 returned error can't find the container with id d81a6edb0aca0287aef3104cdc3304d9aac7cd53792fd69cf75db9b4c06cc254
Sep 10 21:20:06.986703 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:06.986674    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-d9djh"]
Sep 10 21:20:06.988995 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:20:06.988971    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc46f35_6c0b_4406_9578_b94c0a4816c3.slice/crio-03efd64eac91621906ce11b1f2b18b3ef96b2952a20650319a37d6ab9b104dc7 WatchSource:0}: Error finding container 03efd64eac91621906ce11b1f2b18b3ef96b2952a20650319a37d6ab9b104dc7: Status 404 returned error can't find the container with id 03efd64eac91621906ce11b1f2b18b3ef96b2952a20650319a37d6ab9b104dc7
Sep 10 21:20:07.489417 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:07.489382    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-d9djh" event={"ID":"1bc46f35-6c0b-4406-9578-b94c0a4816c3","Type":"ContainerStarted","Data":"03efd64eac91621906ce11b1f2b18b3ef96b2952a20650319a37d6ab9b104dc7"}
Sep 10 21:20:07.491873 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:07.491844    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cdb78488-p6hcg" event={"ID":"5220c899-26dd-4211-86d6-53d3bff44ae1","Type":"ContainerStarted","Data":"d81a6edb0aca0287aef3104cdc3304d9aac7cd53792fd69cf75db9b4c06cc254"}
Sep 10 21:20:09.477060 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.477021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:09.503084 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.503051    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-d9djh" event={"ID":"1bc46f35-6c0b-4406-9578-b94c0a4816c3","Type":"ContainerStarted","Data":"7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc"}
Sep 10 21:20:09.503270 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.503150    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:09.504608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.504585    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cdb78488-p6hcg" event={"ID":"5220c899-26dd-4211-86d6-53d3bff44ae1","Type":"ContainerStarted","Data":"954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8"}
Sep 10 21:20:09.504728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.504626    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:09.525548 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.525489    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77cc466dd6-d9djh" podStartSLOduration=1.206932381 podStartE2EDuration="3.525474981s" totalImagesPullingTime="2.3185426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:09.524078265 +0000 UTC m=+600.051042612" watchObservedRunningTime="2026-09-10 21:20:09.525474981 +0000 UTC m=+600.052439365"
Sep 10 21:20:09.565251 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.565157    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65cdb78488-p6hcg" podStartSLOduration=1.0753540560000001 podStartE2EDuration="3.565141914s" totalImagesPullingTime="2.489787858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:09.56444882 +0000 UTC m=+600.091413167" watchObservedRunningTime="2026-09-10 21:20:09.565141914 +0000 UTC m=+600.092106262"
Sep 10 21:20:09.606781 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.606742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:09.909774 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.909747    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:20:09.910063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:09.910044    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:20:20.511038 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.511005    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:20.512693 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.511089    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:20:20.581670 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.581635    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65cdb78488-p6hcg"]
Sep 10 21:20:20.581834 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.581805    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65cdb78488-p6hcg" podUID="5220c899-26dd-4211-86d6-53d3bff44ae1" containerName="manager" containerID="cri-o://954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8" gracePeriod=30
Sep 10 21:20:20.826174 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.826150    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:20.912638 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.912608    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-metrics-certs\") pod \"5220c899-26dd-4211-86d6-53d3bff44ae1\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") "
Sep 10 21:20:20.912813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.912648    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzvwk\" (UniqueName: \"kubernetes.io/projected/5220c899-26dd-4211-86d6-53d3bff44ae1-kube-api-access-bzvwk\") pod \"5220c899-26dd-4211-86d6-53d3bff44ae1\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") "
Sep 10 21:20:20.912813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.912703    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-webhook-cert\") pod \"5220c899-26dd-4211-86d6-53d3bff44ae1\" (UID: \"5220c899-26dd-4211-86d6-53d3bff44ae1\") "
Sep 10 21:20:20.914879 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.914843    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5220c899-26dd-4211-86d6-53d3bff44ae1" (UID: "5220c899-26dd-4211-86d6-53d3bff44ae1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:20:20.915014 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.914898    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5220c899-26dd-4211-86d6-53d3bff44ae1" (UID: "5220c899-26dd-4211-86d6-53d3bff44ae1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:20:20.915014 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:20.914907    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5220c899-26dd-4211-86d6-53d3bff44ae1-kube-api-access-bzvwk" (OuterVolumeSpecName: "kube-api-access-bzvwk") pod "5220c899-26dd-4211-86d6-53d3bff44ae1" (UID: "5220c899-26dd-4211-86d6-53d3bff44ae1"). InnerVolumeSpecName "kube-api-access-bzvwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:21.013890 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.013858    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-webhook-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:20:21.013890 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.013883    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5220c899-26dd-4211-86d6-53d3bff44ae1-metrics-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:20:21.013890 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.013892    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzvwk\" (UniqueName: \"kubernetes.io/projected/5220c899-26dd-4211-86d6-53d3bff44ae1-kube-api-access-bzvwk\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:20:21.559684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.559647    2583 generic.go:356] "Generic (PLEG): container finished" podID="5220c899-26dd-4211-86d6-53d3bff44ae1" containerID="954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8" exitCode=0
Sep 10 21:20:21.560181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.559695    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65cdb78488-p6hcg"
Sep 10 21:20:21.560181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.559743    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cdb78488-p6hcg" event={"ID":"5220c899-26dd-4211-86d6-53d3bff44ae1","Type":"ContainerDied","Data":"954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8"}
Sep 10 21:20:21.560181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.559796    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65cdb78488-p6hcg" event={"ID":"5220c899-26dd-4211-86d6-53d3bff44ae1","Type":"ContainerDied","Data":"d81a6edb0aca0287aef3104cdc3304d9aac7cd53792fd69cf75db9b4c06cc254"}
Sep 10 21:20:21.560181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.559819    2583 scope.go:117] "RemoveContainer" containerID="954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8"
Sep 10 21:20:21.568805 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.568791    2583 scope.go:117] "RemoveContainer" containerID="954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8"
Sep 10 21:20:21.569059 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:20:21.569039    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8\": container with ID starting with 954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8 not found: ID does not exist" containerID="954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8"
Sep 10 21:20:21.569115 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.569068    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8"} err="failed to get container status \"954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8\": rpc error: code = NotFound desc = could not find container \"954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8\": container with ID starting with 954d89b2396b4588422fc5c8dc957862402579d460829655730b9bbc7c99cec8 not found: ID does not exist"
Sep 10 21:20:21.580630 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.580605    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65cdb78488-p6hcg"]
Sep 10 21:20:21.585235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.585208    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65cdb78488-p6hcg"]
Sep 10 21:20:21.960354 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:21.960319    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5220c899-26dd-4211-86d6-53d3bff44ae1" path="/var/lib/kubelet/pods/5220c899-26dd-4211-86d6-53d3bff44ae1/volumes"
Sep 10 21:20:41.110119 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:41.110017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:41.253461 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:41.253421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:42.708152 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.708118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:42.807710 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.807657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:42.833548 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.833496    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"]
Sep 10 21:20:42.834001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.833987    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5220c899-26dd-4211-86d6-53d3bff44ae1" containerName="manager"
Sep 10 21:20:42.834044 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.834004    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5220c899-26dd-4211-86d6-53d3bff44ae1" containerName="manager"
Sep 10 21:20:42.834082 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.834074    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5220c899-26dd-4211-86d6-53d3bff44ae1" containerName="manager"
Sep 10 21:20:42.837301 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.837282    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:42.839298 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.839276    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:20:42.839429 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.839347    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 21:20:42.839429 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.839347    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-99rzw\""
Sep 10 21:20:42.850220 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.850195    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"]
Sep 10 21:20:42.901949 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.901915    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-maas-api-tls\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:42.902129 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.902103    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:42.902201 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.902173    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12995527-11dd-40ed-b8af-c686dbd2d6fa-openshift-service-ca\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:42.902258 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:42.902205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clqls\" (UniqueName: \"kubernetes.io/projected/12995527-11dd-40ed-b8af-c686dbd2d6fa-kube-api-access-clqls\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.002845 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.002757    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-maas-api-tls\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.002845 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.002833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.003303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.002857    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12995527-11dd-40ed-b8af-c686dbd2d6fa-openshift-service-ca\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.003303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.002875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clqls\" (UniqueName: \"kubernetes.io/projected/12995527-11dd-40ed-b8af-c686dbd2d6fa-kube-api-access-clqls\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.003303 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:20:43.002985    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 21:20:43.003303 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:20:43.003069    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs podName:12995527-11dd-40ed-b8af-c686dbd2d6fa nodeName:}" failed. No retries permitted until 2026-09-10 21:20:43.503046425 +0000 UTC m=+634.030010767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs") pod "maas-api-5b8f865cbb-c4d7q" (UID: "12995527-11dd-40ed-b8af-c686dbd2d6fa") : secret "maas-api-metrics-tls" not found
Sep 10 21:20:43.003571 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.003538    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12995527-11dd-40ed-b8af-c686dbd2d6fa-openshift-service-ca\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.005279 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.005262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-maas-api-tls\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.012569 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.012547    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clqls\" (UniqueName: \"kubernetes.io/projected/12995527-11dd-40ed-b8af-c686dbd2d6fa-kube-api-access-clqls\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.023810 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.023779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:43.221678 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.221649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:43.321866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.321735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:43.337541 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.337498    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"]
Sep 10 21:20:43.348054 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.348029    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.351322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.351299    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 21:20:43.351473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.351329    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g6j29\""
Sep 10 21:20:43.353888 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.353865    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"]
Sep 10 21:20:43.389759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.389732    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"]
Sep 10 21:20:43.399246 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.399220    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.423701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.423679    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"]
Sep 10 21:20:43.507744 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.507710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js6gb\" (UniqueName: \"kubernetes.io/projected/87bc96bd-059c-43de-82d1-3acbdada0f76-kube-api-access-js6gb\") pod \"payload-pre-processing-59fbc47bf8-hz8rz\" (UID: \"87bc96bd-059c-43de-82d1-3acbdada0f76\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.507943 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.507755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c273c2b1-fd37-4dcf-a888-17d3a8438bf9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qzrg6\" (UID: \"c273c2b1-fd37-4dcf-a888-17d3a8438bf9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.507943 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.507845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kznrk\" (UniqueName: \"kubernetes.io/projected/c273c2b1-fd37-4dcf-a888-17d3a8438bf9-kube-api-access-kznrk\") pod \"payload-processing-5b54b7fbb7-qzrg6\" (UID: \"c273c2b1-fd37-4dcf-a888-17d3a8438bf9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.507943 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.507901    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.507943 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.507929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87bc96bd-059c-43de-82d1-3acbdada0f76-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hz8rz\" (UID: \"87bc96bd-059c-43de-82d1-3acbdada0f76\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.511019 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.510998    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs\") pod \"maas-api-5b8f865cbb-c4d7q\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") " pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.608964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.608890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kznrk\" (UniqueName: \"kubernetes.io/projected/c273c2b1-fd37-4dcf-a888-17d3a8438bf9-kube-api-access-kznrk\") pod \"payload-processing-5b54b7fbb7-qzrg6\" (UID: \"c273c2b1-fd37-4dcf-a888-17d3a8438bf9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.608964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.608943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87bc96bd-059c-43de-82d1-3acbdada0f76-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hz8rz\" (UID: \"87bc96bd-059c-43de-82d1-3acbdada0f76\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.609207 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.609037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6gb\" (UniqueName: \"kubernetes.io/projected/87bc96bd-059c-43de-82d1-3acbdada0f76-kube-api-access-js6gb\") pod \"payload-pre-processing-59fbc47bf8-hz8rz\" (UID: \"87bc96bd-059c-43de-82d1-3acbdada0f76\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.609207 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.609060    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c273c2b1-fd37-4dcf-a888-17d3a8438bf9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qzrg6\" (UID: \"c273c2b1-fd37-4dcf-a888-17d3a8438bf9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.609847 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.609827    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c273c2b1-fd37-4dcf-a888-17d3a8438bf9-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-qzrg6\" (UID: \"c273c2b1-fd37-4dcf-a888-17d3a8438bf9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.609985 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.609961    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87bc96bd-059c-43de-82d1-3acbdada0f76-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hz8rz\" (UID: \"87bc96bd-059c-43de-82d1-3acbdada0f76\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.622383 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.622355    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js6gb\" (UniqueName: \"kubernetes.io/projected/87bc96bd-059c-43de-82d1-3acbdada0f76-kube-api-access-js6gb\") pod \"payload-pre-processing-59fbc47bf8-hz8rz\" (UID: \"87bc96bd-059c-43de-82d1-3acbdada0f76\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.622492 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.622355    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kznrk\" (UniqueName: \"kubernetes.io/projected/c273c2b1-fd37-4dcf-a888-17d3a8438bf9-kube-api-access-kznrk\") pod \"payload-processing-5b54b7fbb7-qzrg6\" (UID: \"c273c2b1-fd37-4dcf-a888-17d3a8438bf9\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.660170 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.660142    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:43.708870 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.708832    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:43.748155 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.748125    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:43.945224 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.945182    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"]
Sep 10 21:20:43.948273 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:20:43.948249    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87bc96bd_059c_43de_82d1_3acbdada0f76.slice/crio-adb95e0ba2af61db2d2de3f260ffe935ea6fa25b0176ae70ce385cc087159364 WatchSource:0}: Error finding container adb95e0ba2af61db2d2de3f260ffe935ea6fa25b0176ae70ce385cc087159364: Status 404 returned error can't find the container with id adb95e0ba2af61db2d2de3f260ffe935ea6fa25b0176ae70ce385cc087159364
Sep 10 21:20:43.998397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:43.998373    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"]
Sep 10 21:20:44.001057 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:20:44.001031    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc273c2b1_fd37_4dcf_a888_17d3a8438bf9.slice/crio-bdcfa98992f96fa3574dd005534e82d5680a56b1cb0da89067080ea2808b510f WatchSource:0}: Error finding container bdcfa98992f96fa3574dd005534e82d5680a56b1cb0da89067080ea2808b510f: Status 404 returned error can't find the container with id bdcfa98992f96fa3574dd005534e82d5680a56b1cb0da89067080ea2808b510f
Sep 10 21:20:44.060265 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:44.060232    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"]
Sep 10 21:20:44.067299 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:20:44.067218    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12995527_11dd_40ed_b8af_c686dbd2d6fa.slice/crio-f50a43cf5784df81ed9d62a7acb610ce8317f6511fb8df141255458cb579b2e9 WatchSource:0}: Error finding container f50a43cf5784df81ed9d62a7acb610ce8317f6511fb8df141255458cb579b2e9: Status 404 returned error can't find the container with id f50a43cf5784df81ed9d62a7acb610ce8317f6511fb8df141255458cb579b2e9
Sep 10 21:20:44.664470 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:44.664430    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6" event={"ID":"c273c2b1-fd37-4dcf-a888-17d3a8438bf9","Type":"ContainerStarted","Data":"bdcfa98992f96fa3574dd005534e82d5680a56b1cb0da89067080ea2808b510f"}
Sep 10 21:20:44.666744 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:44.666712    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q" event={"ID":"12995527-11dd-40ed-b8af-c686dbd2d6fa","Type":"ContainerStarted","Data":"f50a43cf5784df81ed9d62a7acb610ce8317f6511fb8df141255458cb579b2e9"}
Sep 10 21:20:44.669000 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:44.668973    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz" event={"ID":"87bc96bd-059c-43de-82d1-3acbdada0f76","Type":"ContainerStarted","Data":"adb95e0ba2af61db2d2de3f260ffe935ea6fa25b0176ae70ce385cc087159364"}
Sep 10 21:20:47.174246 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:47.174174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:48.694306 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.694271    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q" event={"ID":"12995527-11dd-40ed-b8af-c686dbd2d6fa","Type":"ContainerStarted","Data":"99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4"}
Sep 10 21:20:48.694782 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.694365    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:48.695916 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.695891    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz" event={"ID":"87bc96bd-059c-43de-82d1-3acbdada0f76","Type":"ContainerStarted","Data":"9ef05325ac7221fc117501a06ace9867bf331a1c29f3c4a2a451eab7473ce962"}
Sep 10 21:20:48.696033 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.695984    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:48.697439 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.697407    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6" event={"ID":"c273c2b1-fd37-4dcf-a888-17d3a8438bf9","Type":"ContainerStarted","Data":"8bcd06535e3e1bfd9eabe2305921568084a44fa0d519ddfe0aa276f18c516c25"}
Sep 10 21:20:48.697580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.697468    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:20:48.733130 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.733059    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q" podStartSLOduration=3.040436279 podStartE2EDuration="6.733024871s" totalImagesPullingTime="3.692588592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:48.712364559 +0000 UTC m=+639.239328907" watchObservedRunningTime="2026-09-10 21:20:48.733024871 +0000 UTC m=+639.259989219"
Sep 10 21:20:48.733331 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.733217    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6" podStartSLOduration=1.9338678040000001 podStartE2EDuration="5.733209255s" totalImagesPullingTime="3.799341451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:48.731686646 +0000 UTC m=+639.258650994" watchObservedRunningTime="2026-09-10 21:20:48.733209255 +0000 UTC m=+639.260173600"
Sep 10 21:20:48.747431 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:48.747375    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz" podStartSLOduration=1.862315459 podStartE2EDuration="5.747358846s" totalImagesPullingTime="3.885043387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:48.745184046 +0000 UTC m=+639.272148393" watchObservedRunningTime="2026-09-10 21:20:48.747358846 +0000 UTC m=+639.274323193"
Sep 10 21:20:52.723317 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:52.723239    2583 generic.go:356] "Generic (PLEG): container finished" podID="ad5f6765-9edd-47fc-a141-d780d3aff1af" containerID="f2d596b0f7eda003b873416372e7b1711b574083e9bf4157c82ffc1317acb8c8" exitCode=0
Sep 10 21:20:52.723317 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:52.723292    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" event={"ID":"ad5f6765-9edd-47fc-a141-d780d3aff1af","Type":"ContainerDied","Data":"f2d596b0f7eda003b873416372e7b1711b574083e9bf4157c82ffc1317acb8c8"}
Sep 10 21:20:52.723767 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:52.723670    2583 scope.go:117] "RemoveContainer" containerID="f2d596b0f7eda003b873416372e7b1711b574083e9bf4157c82ffc1317acb8c8"
Sep 10 21:20:52.724705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:52.724678    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:20:52.724786 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:52.724760    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:20:52.724823 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:52.724805    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:20:53.732488 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:53.732454    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" event={"ID":"ad5f6765-9edd-47fc-a141-d780d3aff1af","Type":"ContainerStarted","Data":"46c396974c56eb4592dab5fad29f1b1228576d51454851d726b95ca1dcc86e84"}
Sep 10 21:20:53.756483 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:53.756449    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:20:53.756679 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:53.756652    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dc9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 10 21:20:53.756732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:53.756710    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" podUID="ad5f6765-9edd-47fc-a141-d780d3aff1af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 10 21:20:54.716302 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:54.716267    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:20:54.757127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:54.757097    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dc9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 10 21:20:54.757516 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:54.757147    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" podUID="ad5f6765-9edd-47fc-a141-d780d3aff1af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 10 21:20:55.697355 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:55.697321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:55.757102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:55.757068    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dc9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 10 21:20:55.757277 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:55.757122    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" podUID="ad5f6765-9edd-47fc-a141-d780d3aff1af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 10 21:20:55.797756 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:55.797721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:56.756699 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:56.756668    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dc9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 10 21:20:56.756866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:56.756720    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" podUID="ad5f6765-9edd-47fc-a141-d780d3aff1af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 10 21:20:57.757270 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:57.757237    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dc9wf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 10 21:20:57.757687 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:57.757294    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf" podUID="ad5f6765-9edd-47fc-a141-d780d3aff1af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 10 21:20:58.761703 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:58.761675    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:20:58.762091 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:58.761951    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:20:58.762729 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:58.762710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dc9wf"
Sep 10 21:20:59.711031 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:59.711002    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hz8rz"
Sep 10 21:20:59.711196 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:20:59.711178    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-qzrg6"
Sep 10 21:21:03.640691 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:03.640658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:11.441466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:11.441432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:11.543457 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:11.543424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:23.192349 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.192312    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"]
Sep 10 21:21:23.197540 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.197517    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.200158 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.200133    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 21:21:23.200158 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.200146    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sprxz\""
Sep 10 21:21:23.200358 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.200162    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 21:21:23.200358 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.200133    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 21:21:23.207028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.207005    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"]
Sep 10 21:21:23.246738 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.246709    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:23.269923 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.269896    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.270052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.269936    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.270052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.269956    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.270167 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.270055    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.270167 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.270126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.270236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.270175    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzqnt\" (UniqueName: \"kubernetes.io/projected/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-kube-api-access-vzqnt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.371623 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.371592    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.371825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.371664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.371825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.371698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzqnt\" (UniqueName: \"kubernetes.io/projected/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-kube-api-access-vzqnt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.371825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.371747    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.371825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.371793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.372054 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.371944    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.372054 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.372024    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.372168 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.372119    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.372258 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.372241    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.373976 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.373955    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.374280 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.374262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.379159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.379140    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzqnt\" (UniqueName: \"kubernetes.io/projected/1ac8388d-fdbd-4f31-815c-aa514dfeabaa-kube-api-access-vzqnt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c249s\" (UID: \"1ac8388d-fdbd-4f31-815c-aa514dfeabaa\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.511057 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.510978    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:23.789147 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.789070    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"]
Sep 10 21:21:23.792209 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:21:23.792176    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac8388d_fdbd_4f31_815c_aa514dfeabaa.slice/crio-e05bcfadfc499b72af626efd96498ce5a2ab3436a6048028687017252dd73813 WatchSource:0}: Error finding container e05bcfadfc499b72af626efd96498ce5a2ab3436a6048028687017252dd73813: Status 404 returned error can't find the container with id e05bcfadfc499b72af626efd96498ce5a2ab3436a6048028687017252dd73813
Sep 10 21:21:23.875058 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:23.875024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s" event={"ID":"1ac8388d-fdbd-4f31-815c-aa514dfeabaa","Type":"ContainerStarted","Data":"e05bcfadfc499b72af626efd96498ce5a2ab3436a6048028687017252dd73813"}
Sep 10 21:21:24.881935 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:24.881902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s" event={"ID":"1ac8388d-fdbd-4f31-815c-aa514dfeabaa","Type":"ContainerStarted","Data":"41c8623c96c8a36dc21c2cdffa0e9257a40870990061e9dbd9e162689e59ca23"}
Sep 10 21:21:24.898622 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:24.898576    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s" podStartSLOduration=0.950751742 podStartE2EDuration="1.898559659s" totalImagesPullingTime="947.807917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:24.898072383 +0000 UTC m=+675.425036730" watchObservedRunningTime="2026-09-10 21:21:24.898559659 +0000 UTC m=+675.425524008"
Sep 10 21:21:26.043540 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:26.043472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:29.761756 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:29.761724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:33.511406 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:33.511372    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:33.511815 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:33.511517    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:33.523732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:33.523715    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:33.935102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:33.935071    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c249s"
Sep 10 21:21:34.636092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:34.636058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:42.128990 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.128955    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"]
Sep 10 21:21:42.131956 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.131934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.134294 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.134271    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 21:21:42.143172 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.143148    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"]
Sep 10 21:21:42.156094 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.156065    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:42.239977 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.239940    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df849b4-50d7-4311-92c8-68f559f606c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.240148 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.239983    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.240148 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.240046    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmxsp\" (UniqueName: \"kubernetes.io/projected/0df849b4-50d7-4311-92c8-68f559f606c8-kube-api-access-fmxsp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.240148 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.240067    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.240148 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.240129    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.240297 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.240204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.341647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.341609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmxsp\" (UniqueName: \"kubernetes.io/projected/0df849b4-50d7-4311-92c8-68f559f606c8-kube-api-access-fmxsp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.341647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.341648    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.341893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.341671    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.341893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.341712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.341893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.341762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df849b4-50d7-4311-92c8-68f559f606c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.341893 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.341795    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.342187 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.342162    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.342254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.342172    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.342254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.342198    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.343934 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.343911    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df849b4-50d7-4311-92c8-68f559f606c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.344253 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.344232    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df849b4-50d7-4311-92c8-68f559f606c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.349603 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.349581    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmxsp\" (UniqueName: \"kubernetes.io/projected/0df849b4-50d7-4311-92c8-68f559f606c8-kube-api-access-fmxsp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j\" (UID: \"0df849b4-50d7-4311-92c8-68f559f606c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.443757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.443715    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:42.716269 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.716246    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"]
Sep 10 21:21:42.719534 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:21:42.719493    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df849b4_50d7_4311_92c8_68f559f606c8.slice/crio-d3147bc28664e8df8837b9b8240f724248811e08962f08ab3d3fb0b216b203d4 WatchSource:0}: Error finding container d3147bc28664e8df8837b9b8240f724248811e08962f08ab3d3fb0b216b203d4: Status 404 returned error can't find the container with id d3147bc28664e8df8837b9b8240f724248811e08962f08ab3d3fb0b216b203d4
Sep 10 21:21:42.720928 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.720911    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:21:42.965048 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.965018    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j" event={"ID":"0df849b4-50d7-4311-92c8-68f559f606c8","Type":"ContainerStarted","Data":"dd9b10d0ca92a730ddf935db97e91f0e29912c4c9e2461c324855a0bc57de6da"}
Sep 10 21:21:42.965191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.965068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j" event={"ID":"0df849b4-50d7-4311-92c8-68f559f606c8","Type":"ContainerStarted","Data":"d3147bc28664e8df8837b9b8240f724248811e08962f08ab3d3fb0b216b203d4"}
Sep 10 21:21:42.981327 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:42.981218    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j" podStartSLOduration=0.824416703 podStartE2EDuration="981.196709ms" totalImagesPullingTime="156.780006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:42.980748076 +0000 UTC m=+693.507712424" watchObservedRunningTime="2026-09-10 21:21:42.981196709 +0000 UTC m=+693.508161061"
Sep 10 21:21:52.444385 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:52.444344    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:52.444883 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:52.444435    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:52.457156 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:52.457133    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:53.022959 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:53.022930    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j"
Sep 10 21:21:54.115136 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:54.115103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:58.564958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:21:58.564927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:03.619637 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:03.619605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:03.720159 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:03.720061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:39.878334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:39.878297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:39.965670 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:39.965638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:54.859911 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:54.859862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:54.964120 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:54.964085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:56.066608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:56.066564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:56.147885 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:22:56.147849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:11.251254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.251212    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:11.384970 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.384935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:11.631456 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.631364    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"]
Sep 10 21:23:11.634335 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.634311    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.636204 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.636175    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-5ec041-openshift-default-dockercfg-sr25q\""
Sep 10 21:23:11.650739 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.650712    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"]
Sep 10 21:23:11.655058 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-credential-socket\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655203 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-certs\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655203 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655093    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-envoy\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655331 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnjlp\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-kube-api-access-fnjlp\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655331 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655229    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-socket\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655331 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-token\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655331 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655312    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c074bb-314e-43e4-80d9-0fbdb8eef015-istiod-ca-cert\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-podinfo\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.655577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.655381    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-data\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.665358 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.665314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:11.756375 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-data\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756448    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-credential-socket\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-certs\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756499    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-envoy\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnjlp\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-kube-api-access-fnjlp\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-socket\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-token\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c074bb-314e-43e4-80d9-0fbdb8eef015-istiod-ca-cert\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756736    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-data\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.756871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756846    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-certs\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.757046 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.756890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-podinfo\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.757525 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.757487    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-credential-socket\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.757630 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.757582    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-socket\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.757802 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.757783    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c074bb-314e-43e4-80d9-0fbdb8eef015-istiod-ca-cert\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.759162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.759139    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-envoy\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.759629 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.759612    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-podinfo\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.767975 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.767954    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-token\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.768263 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.768245    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnjlp\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-kube-api-access-fnjlp\") pod \"e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") " pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.947547 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.947497    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:11.998906 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:11.998840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:12.232305 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.232278    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"]
Sep 10 21:23:12.234719 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:12.234690    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c074bb_314e_43e4_80d9_0fbdb8eef015.slice/crio-cdc69c83148c6adbab833ee6e3e18a5a1eafb5d267e1118ae024b28da1da3255 WatchSource:0}: Error finding container cdc69c83148c6adbab833ee6e3e18a5a1eafb5d267e1118ae024b28da1da3255: Status 404 returned error can't find the container with id cdc69c83148c6adbab833ee6e3e18a5a1eafb5d267e1118ae024b28da1da3255
Sep 10 21:23:12.236865 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.236835    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:23:12.236964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.236911    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:23:12.236964 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.236956    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:23:12.382645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.382605    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" event={"ID":"32c074bb-314e-43e4-80d9-0fbdb8eef015","Type":"ContainerStarted","Data":"495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4"}
Sep 10 21:23:12.383047 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.382664    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" event={"ID":"32c074bb-314e-43e4-80d9-0fbdb8eef015","Type":"ContainerStarted","Data":"cdc69c83148c6adbab833ee6e3e18a5a1eafb5d267e1118ae024b28da1da3255"}
Sep 10 21:23:12.407614 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.407562    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podStartSLOduration=1.407545848 podStartE2EDuration="1.407545848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:12.405475864 +0000 UTC m=+782.932440247" watchObservedRunningTime="2026-09-10 21:23:12.407545848 +0000 UTC m=+782.934510194"
Sep 10 21:23:12.593752 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.593665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:12.689851 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.689810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:12.947817 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:12.947788    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:13.012680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:13.012647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:13.953004 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:13.952969    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:14.282838 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:14.282753    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:14.383292 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:14.383257    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:14.393795 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:14.393767    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:14.394573 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:14.394553    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:15.434356 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.434321    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"]
Sep 10 21:23:15.445188 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.445143    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.450249 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.449775    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-5ec041-serving-cert\""
Sep 10 21:23:15.451243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.451183    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"]
Sep 10 21:23:15.474231 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.474200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:15.494667 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.494629    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmd6x\" (UniqueName: \"kubernetes.io/projected/b8173489-ac09-4b21-a110-866bd9844723-kube-api-access-pmd6x\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.494848 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.494730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8173489-ac09-4b21-a110-866bd9844723-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.494848 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.494760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-metrics-certs\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.494848 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.494827    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.571334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.571298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:15.595574 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.595538    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.595763 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.595628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmd6x\" (UniqueName: \"kubernetes.io/projected/b8173489-ac09-4b21-a110-866bd9844723-kube-api-access-pmd6x\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.595763 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.595664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8173489-ac09-4b21-a110-866bd9844723-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.595763 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.595683    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-metrics-certs\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.595763 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:23:15.595683    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-serving-cert: secret "maas-api-e2e-inf-a-5ec041-serving-cert" not found
Sep 10 21:23:15.595763 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:23:15.595767    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls podName:b8173489-ac09-4b21-a110-866bd9844723 nodeName:}" failed. No retries permitted until 2026-09-10 21:23:16.095746396 +0000 UTC m=+786.622710738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls") pod "maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n" (UID: "b8173489-ac09-4b21-a110-866bd9844723") : secret "maas-api-e2e-inf-a-5ec041-serving-cert" not found
Sep 10 21:23:15.596421 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.596400    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8173489-ac09-4b21-a110-866bd9844723-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.598108 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.598086    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-metrics-certs\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.606554 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.606531    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmd6x\" (UniqueName: \"kubernetes.io/projected/b8173489-ac09-4b21-a110-866bd9844723-kube-api-access-pmd6x\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:15.736902 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:15.736811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:16.101655 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.101566    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:16.103919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.103890    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls\") pod \"maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:16.112659 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.112631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:16.212880 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.212844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:16.235912 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.235885    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"]
Sep 10 21:23:16.238600 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.238578    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.240438 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.240420    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-5ec041\""
Sep 10 21:23:16.240537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.240465    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-5ec041-dockercfg-mnhg6\""
Sep 10 21:23:16.247466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.247444    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"]
Sep 10 21:23:16.250242 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.250221    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.250881 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.250862    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"]
Sep 10 21:23:16.260380 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.260324    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"]
Sep 10 21:23:16.303788 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.303755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trklh\" (UniqueName: \"kubernetes.io/projected/67ca510d-5611-4d2b-859e-497df0cfb0f8-kube-api-access-trklh\") pod \"payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.303980 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.303797    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ae1169-2e9e-4af6-be49-e61222866bd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.303980 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.303881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67ca510d-5611-4d2b-859e-497df0cfb0f8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.303980 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.303913    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68rrp\" (UniqueName: \"kubernetes.io/projected/e5ae1169-2e9e-4af6-be49-e61222866bd7-kube-api-access-68rrp\") pod \"payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.359933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.359841    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:16.404787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.404755    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67ca510d-5611-4d2b-859e-497df0cfb0f8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.404787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.404790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68rrp\" (UniqueName: \"kubernetes.io/projected/e5ae1169-2e9e-4af6-be49-e61222866bd7-kube-api-access-68rrp\") pod \"payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.405052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.404865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trklh\" (UniqueName: \"kubernetes.io/projected/67ca510d-5611-4d2b-859e-497df0cfb0f8-kube-api-access-trklh\") pod \"payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.405052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.404888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ae1169-2e9e-4af6-be49-e61222866bd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.405548 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.405525    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ae1169-2e9e-4af6-be49-e61222866bd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.405759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.405730    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67ca510d-5611-4d2b-859e-497df0cfb0f8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.413752 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.413720    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68rrp\" (UniqueName: \"kubernetes.io/projected/e5ae1169-2e9e-4af6-be49-e61222866bd7-kube-api-access-68rrp\") pod \"payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.417818 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.415943    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trklh\" (UniqueName: \"kubernetes.io/projected/67ca510d-5611-4d2b-859e-497df0cfb0f8-kube-api-access-trklh\") pod \"payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.514484 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.514448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:16.550414 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.550375    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:16.563163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.563135    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:16.647216 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.647167    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"]
Sep 10 21:23:16.651219 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:16.651190    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8173489_ac09_4b21_a110_866bd9844723.slice/crio-6bf2a2e8bebcb4d4707f03a938a9cb1d42ba2c5d66e9e7ebf79eed5790a63e8d WatchSource:0}: Error finding container 6bf2a2e8bebcb4d4707f03a938a9cb1d42ba2c5d66e9e7ebf79eed5790a63e8d: Status 404 returned error can't find the container with id 6bf2a2e8bebcb4d4707f03a938a9cb1d42ba2c5d66e9e7ebf79eed5790a63e8d
Sep 10 21:23:16.832127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.832093    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"]
Sep 10 21:23:16.835037 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:16.835002    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ae1169_2e9e_4af6_be49_e61222866bd7.slice/crio-1e74dba1cbd72eba51de9165ba5ed9feb4bf65245590c5a2ec6b0e6137a46029 WatchSource:0}: Error finding container 1e74dba1cbd72eba51de9165ba5ed9feb4bf65245590c5a2ec6b0e6137a46029: Status 404 returned error can't find the container with id 1e74dba1cbd72eba51de9165ba5ed9feb4bf65245590c5a2ec6b0e6137a46029
Sep 10 21:23:16.866528 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:16.866144    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"]
Sep 10 21:23:16.871792 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:16.871363    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ca510d_5611_4d2b_859e_497df0cfb0f8.slice/crio-0a6fdab376874ab0a811f86c561ce1b882e205d26c8362fef3b5c4db67b42e5d WatchSource:0}: Error finding container 0a6fdab376874ab0a811f86c561ce1b882e205d26c8362fef3b5c4db67b42e5d: Status 404 returned error can't find the container with id 0a6fdab376874ab0a811f86c561ce1b882e205d26c8362fef3b5c4db67b42e5d
Sep 10 21:23:17.414141 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.414102    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn" event={"ID":"e5ae1169-2e9e-4af6-be49-e61222866bd7","Type":"ContainerStarted","Data":"b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d"}
Sep 10 21:23:17.414381 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.414171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn" event={"ID":"e5ae1169-2e9e-4af6-be49-e61222866bd7","Type":"ContainerStarted","Data":"1e74dba1cbd72eba51de9165ba5ed9feb4bf65245590c5a2ec6b0e6137a46029"}
Sep 10 21:23:17.414381 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.414260    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:17.415913 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.415883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n" event={"ID":"b8173489-ac09-4b21-a110-866bd9844723","Type":"ContainerStarted","Data":"6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a"}
Sep 10 21:23:17.416048 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.415934    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n" event={"ID":"b8173489-ac09-4b21-a110-866bd9844723","Type":"ContainerStarted","Data":"6bf2a2e8bebcb4d4707f03a938a9cb1d42ba2c5d66e9e7ebf79eed5790a63e8d"}
Sep 10 21:23:17.416048 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.415991    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:17.417577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.417556    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r" event={"ID":"67ca510d-5611-4d2b-859e-497df0cfb0f8","Type":"ContainerStarted","Data":"d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214"}
Sep 10 21:23:17.417680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.417591    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r" event={"ID":"67ca510d-5611-4d2b-859e-497df0cfb0f8","Type":"ContainerStarted","Data":"0a6fdab376874ab0a811f86c561ce1b882e205d26c8362fef3b5c4db67b42e5d"}
Sep 10 21:23:17.417746 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.417678    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:17.432805 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.432752    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn" podStartSLOduration=1.432731949 podStartE2EDuration="1.432731949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:17.431624147 +0000 UTC m=+787.958588494" watchObservedRunningTime="2026-09-10 21:23:17.432731949 +0000 UTC m=+787.959696298"
Sep 10 21:23:17.456406 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.456351    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n" podStartSLOduration=2.009302861 podStartE2EDuration="2.456336142s" totalImagesPullingTime="447.033281ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:17.455158287 +0000 UTC m=+787.982122669" watchObservedRunningTime="2026-09-10 21:23:17.456336142 +0000 UTC m=+787.983300488"
Sep 10 21:23:17.474320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:17.474264    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r" podStartSLOduration=1.474247721 podStartE2EDuration="1.474247721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:17.47149728 +0000 UTC m=+787.998461632" watchObservedRunningTime="2026-09-10 21:23:17.474247721 +0000 UTC m=+788.001212069"
Sep 10 21:23:18.857920 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:18.857882    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:18.930872 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:18.930840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.079220 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:19.079183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.399190 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:19.399151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.529758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:19.529726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.707192 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:19.707097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.859678 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:19.859643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.949805 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:19.949762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:21.523100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:21.523066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:23.431410 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:23.431378    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:23:24.832976 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:24.832938    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:24.925248 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:24.925208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:25.390859 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:25.390815    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:25.496104 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:25.496066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.053201 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.053150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.150892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.150854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.248309 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.248269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.684182 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.684146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.783577 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.783543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.869632 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.869596    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"]
Sep 10 21:23:26.874426 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.874404    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:26.876689 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.876667    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-780a0cdd-dockercfg-kwl5z\""
Sep 10 21:23:26.876832 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.876693    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-780a0cdd\""
Sep 10 21:23:26.886300 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:26.886273    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"]
Sep 10 21:23:27.007324 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.007229    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6w4w\" (UniqueName: \"kubernetes.io/projected/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-kube-api-access-t6w4w\") pod \"payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:27.007974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.007949    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:27.110061 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.110020    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6w4w\" (UniqueName: \"kubernetes.io/projected/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-kube-api-access-t6w4w\") pod \"payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:27.111043 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.111017    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:27.112315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.112109    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:27.118783 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.118754    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6w4w\" (UniqueName: \"kubernetes.io/projected/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-kube-api-access-t6w4w\") pod \"payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:27.165432 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.165400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.185360 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.185319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:27.265918 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.265815    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.497719 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.497688    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"]
Sep 10 21:23:27.500789 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:27.500449    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467f5aa8_6387_4aaa_9a12_cbebe7c2b9ab.slice/crio-860b2fba196b590a6f527d32236550eb169c2f2afe5974bc9d68d8bf313d92f7 WatchSource:0}: Error finding container 860b2fba196b590a6f527d32236550eb169c2f2afe5974bc9d68d8bf313d92f7: Status 404 returned error can't find the container with id 860b2fba196b590a6f527d32236550eb169c2f2afe5974bc9d68d8bf313d92f7
Sep 10 21:23:27.636924 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.636888    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.741672 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.741641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.837256 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.837170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.971405 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.971377    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"]
Sep 10 21:23:27.975305 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.975281    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:27.977352 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.977172    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-f135f4-dockercfg-v26tn\""
Sep 10 21:23:27.977352 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.977275    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-f135f4\""
Sep 10 21:23:27.985618 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:27.985590    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"]
Sep 10 21:23:28.021540 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.020548    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"]
Sep 10 21:23:28.025310 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.025280    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.050051 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.049944    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"]
Sep 10 21:23:28.147532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.147467    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5vr9\" (UniqueName: \"kubernetes.io/projected/dc5097ce-28f6-422c-9495-92ad3572848b-kube-api-access-m5vr9\") pod \"payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") " pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.147532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.147532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/671be250-e7b0-4c47-aaf2-83fd141fe9ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:28.148213 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.147575    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxgb4\" (UniqueName: \"kubernetes.io/projected/671be250-e7b0-4c47-aaf2-83fd141fe9ca-kube-api-access-bxgb4\") pod \"payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:28.148213 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.147703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5097ce-28f6-422c-9495-92ad3572848b-plugins-config-volume\") pod \"payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") " pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.249145 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.249111    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/671be250-e7b0-4c47-aaf2-83fd141fe9ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:28.249296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.249160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxgb4\" (UniqueName: \"kubernetes.io/projected/671be250-e7b0-4c47-aaf2-83fd141fe9ca-kube-api-access-bxgb4\") pod \"payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:28.249296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.249223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5097ce-28f6-422c-9495-92ad3572848b-plugins-config-volume\") pod \"payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") " pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.249397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.249316    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vr9\" (UniqueName: \"kubernetes.io/projected/dc5097ce-28f6-422c-9495-92ad3572848b-kube-api-access-m5vr9\") pod \"payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") " pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.249932 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.249913    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/671be250-e7b0-4c47-aaf2-83fd141fe9ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:28.250252 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.250231    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5097ce-28f6-422c-9495-92ad3572848b-plugins-config-volume\") pod \"payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") " pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.257705 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.257678    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxgb4\" (UniqueName: \"kubernetes.io/projected/671be250-e7b0-4c47-aaf2-83fd141fe9ca-kube-api-access-bxgb4\") pod \"payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:28.257856 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.257838    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5vr9\" (UniqueName: \"kubernetes.io/projected/dc5097ce-28f6-422c-9495-92ad3572848b-kube-api-access-m5vr9\") pod \"payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") " pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.286709 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.286674    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:28.296989 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.296960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:28.351132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.350600    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:28.425996 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.425930    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:23:28.429443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.428900    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:23:28.481665 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.481628    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph" event={"ID":"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab","Type":"ContainerStarted","Data":"93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf"}
Sep 10 21:23:28.481948 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.481928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph" event={"ID":"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab","Type":"ContainerStarted","Data":"860b2fba196b590a6f527d32236550eb169c2f2afe5974bc9d68d8bf313d92f7"}
Sep 10 21:23:28.482075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.482053    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:28.499805 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.499759    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph" podStartSLOduration=2.499741371 podStartE2EDuration="2.499741371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:28.497855803 +0000 UTC m=+799.024820176" watchObservedRunningTime="2026-09-10 21:23:28.499741371 +0000 UTC m=+799.026705718"
Sep 10 21:23:28.564845 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.564754    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"]
Sep 10 21:23:28.568130 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:28.568100    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671be250_e7b0_4c47_aaf2_83fd141fe9ca.slice/crio-aa9e1dbfdba0988d3ce626b16175cf89db036455df04f52231dd93b6faac2d9e WatchSource:0}: Error finding container aa9e1dbfdba0988d3ce626b16175cf89db036455df04f52231dd93b6faac2d9e: Status 404 returned error can't find the container with id aa9e1dbfdba0988d3ce626b16175cf89db036455df04f52231dd93b6faac2d9e
Sep 10 21:23:28.644071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:28.644041    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"]
Sep 10 21:23:28.646194 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:28.646171    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5097ce_28f6_422c_9495_92ad3572848b.slice/crio-fa3fde0021cfa2b456f988c0acad7d16be6b909b29ac191343c96b5795d3e563 WatchSource:0}: Error finding container fa3fde0021cfa2b456f988c0acad7d16be6b909b29ac191343c96b5795d3e563: Status 404 returned error can't find the container with id fa3fde0021cfa2b456f988c0acad7d16be6b909b29ac191343c96b5795d3e563
Sep 10 21:23:29.197895 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.197849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:29.488635 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.488538    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz" event={"ID":"dc5097ce-28f6-422c-9495-92ad3572848b","Type":"ContainerStarted","Data":"f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24"}
Sep 10 21:23:29.488635 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.488606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz" event={"ID":"dc5097ce-28f6-422c-9495-92ad3572848b","Type":"ContainerStarted","Data":"fa3fde0021cfa2b456f988c0acad7d16be6b909b29ac191343c96b5795d3e563"}
Sep 10 21:23:29.488635 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.488635    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:29.490099 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.490068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d" event={"ID":"671be250-e7b0-4c47-aaf2-83fd141fe9ca","Type":"ContainerStarted","Data":"a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6"}
Sep 10 21:23:29.490234 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.490118    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d" event={"ID":"671be250-e7b0-4c47-aaf2-83fd141fe9ca","Type":"ContainerStarted","Data":"aa9e1dbfdba0988d3ce626b16175cf89db036455df04f52231dd93b6faac2d9e"}
Sep 10 21:23:29.505125 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.504859    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz" podStartSLOduration=1.504839172 podStartE2EDuration="1.504839172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:29.504435802 +0000 UTC m=+800.031400152" watchObservedRunningTime="2026-09-10 21:23:29.504839172 +0000 UTC m=+800.031803525"
Sep 10 21:23:29.520353 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:29.520287    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d" podStartSLOduration=2.520270377 podStartE2EDuration="2.520270377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:29.518697219 +0000 UTC m=+800.045661569" watchObservedRunningTime="2026-09-10 21:23:29.520270377 +0000 UTC m=+800.047234724"
Sep 10 21:23:30.237826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:30.237790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:30.398453 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:30.398421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:30.436695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:30.436644    2583 patch_prober.go:28] interesting pod/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:23:30.436871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:30.436714    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:23:30.500090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:30.500005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:30.500738 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:30.500710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:30.673559 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:30.673494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:31.506677 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:31.506595    2583 generic.go:356] "Generic (PLEG): container finished" podID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerID="495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4" exitCode=0
Sep 10 21:23:31.507091 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:31.506669    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" event={"ID":"32c074bb-314e-43e4-80d9-0fbdb8eef015","Type":"ContainerDied","Data":"495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4"}
Sep 10 21:23:31.507209 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:31.507194    2583 scope.go:117] "RemoveContainer" containerID="495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4"
Sep 10 21:23:31.508228 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:31.508190    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:23:31.508286 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:31.508269    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:23:31.508327 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:31.508300    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:23:32.518142 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:32.518036    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" event={"ID":"32c074bb-314e-43e4-80d9-0fbdb8eef015","Type":"ContainerStarted","Data":"2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea"}
Sep 10 21:23:32.948695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:32.948661    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:32.948925 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:32.948902    2583 patch_prober.go:28] interesting pod/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:23:32.948981 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:32.948953    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 10 21:23:33.267588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:33.267490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:33.948776 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:33.948743    2583 patch_prober.go:28] interesting pod/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:23:33.949171 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:33.948795    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 10 21:23:34.568966 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:34.568925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:34.715121 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:34.715088    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:34.813327 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:34.813295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:34.914625 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:34.914578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:34.948783 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:34.948745    2583 patch_prober.go:28] interesting pod/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:23:34.949241 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:34.948805    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 10 21:23:35.948828 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:35.948784    2583 patch_prober.go:28] interesting pod/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:23:35.949229 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:35.948847    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 10 21:23:36.574813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:36.574781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:36.948067 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:36.948033    2583 patch_prober.go:28] interesting pod/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:23:36.948254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:36.948090    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 10 21:23:37.951775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:37.951740    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:37.952163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:37.952051    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:37.952787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:37.952767    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:23:39.166926 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:39.166884    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:39.492736 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:39.492648    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:23:39.496463 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:39.496435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:39.843132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:39.843044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:39.952160 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:39.952127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:40.503776 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:40.503742    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:23:41.242184 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:41.242150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:41.346711 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:41.346674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:41.442238 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:41.442199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:41.508317 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:41.508245    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:23:41.583100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:41.583068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:42.569564 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:42.569517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:42.667068 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:42.667033    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:43.760004 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:43.759969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:43.861744 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:43.861713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:44.380644 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.380611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:44.481478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.481416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:44.574118 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.574083    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"]
Sep 10 21:23:44.577884 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.577864    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:44.581238 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.581215    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a6441b-dockercfg-gk4kt\""
Sep 10 21:23:44.581590 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.581561    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a6441b\""
Sep 10 21:23:44.589091 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.589068    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"]
Sep 10 21:23:44.620579 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.620533    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf8kb\" (UniqueName: \"kubernetes.io/projected/db00935c-7a76-4be4-b060-650ad8de0dc9-kube-api-access-vf8kb\") pod \"payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:44.620759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.620602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db00935c-7a76-4be4-b060-650ad8de0dc9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:44.624764 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.624736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:44.722055 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.722024    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf8kb\" (UniqueName: \"kubernetes.io/projected/db00935c-7a76-4be4-b060-650ad8de0dc9-kube-api-access-vf8kb\") pod \"payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:44.722277 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.722068    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db00935c-7a76-4be4-b060-650ad8de0dc9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:44.722841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.722819    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db00935c-7a76-4be4-b060-650ad8de0dc9-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:44.730397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.730369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf8kb\" (UniqueName: \"kubernetes.io/projected/db00935c-7a76-4be4-b060-650ad8de0dc9-kube-api-access-vf8kb\") pod \"payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:44.891321 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:44.891283    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:45.164120 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:45.164094    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"]
Sep 10 21:23:45.166017 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:23:45.165980    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb00935c_7a76_4be4_b060_650ad8de0dc9.slice/crio-5245ba2130608f930642c6fc143f36e3293c9fab3b598a1f50855c5b1b756d63 WatchSource:0}: Error finding container 5245ba2130608f930642c6fc143f36e3293c9fab3b598a1f50855c5b1b756d63: Status 404 returned error can't find the container with id 5245ba2130608f930642c6fc143f36e3293c9fab3b598a1f50855c5b1b756d63
Sep 10 21:23:45.591807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:45.591717    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4" event={"ID":"db00935c-7a76-4be4-b060-650ad8de0dc9","Type":"ContainerStarted","Data":"4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576"}
Sep 10 21:23:45.591807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:45.591781    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4" event={"ID":"db00935c-7a76-4be4-b060-650ad8de0dc9","Type":"ContainerStarted","Data":"5245ba2130608f930642c6fc143f36e3293c9fab3b598a1f50855c5b1b756d63"}
Sep 10 21:23:45.591997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:45.591921    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:23:45.608978 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:45.608918    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4" podStartSLOduration=1.6088964959999998 podStartE2EDuration="1.608896496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:45.606395021 +0000 UTC m=+816.133359358" watchObservedRunningTime="2026-09-10 21:23:45.608896496 +0000 UTC m=+816.135860845"
Sep 10 21:23:47.217785 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:47.217741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:47.806596 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:47.806539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:47.908086 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:47.908050    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:48.594409 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:48.594370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:49.516903 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:49.516863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:50.248314 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:50.248281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:50.325030 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:50.324998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:51.110337 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:51.110295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:56.599583 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:23:56.599551    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:24:04.960552 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:04.960513    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:06.856063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:06.856031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:06.893062 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:06.893026    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"]
Sep 10 21:24:06.893315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:06.893290    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph" podUID="467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" containerName="payload-pre-processing" containerID="cri-o://93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf" gracePeriod=30
Sep 10 21:24:06.957907 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:06.957841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:07.055656 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.055620    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:07.150584 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.150556    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:24:07.155891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.155844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:07.245634 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.245600    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6w4w\" (UniqueName: \"kubernetes.io/projected/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-kube-api-access-t6w4w\") pod \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") "
Sep 10 21:24:07.245806 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.245726    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-plugins-config-volume\") pod \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\" (UID: \"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab\") "
Sep 10 21:24:07.246030 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.246008    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" (UID: "467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:07.247749 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.247729    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-kube-api-access-t6w4w" (OuterVolumeSpecName: "kube-api-access-t6w4w") pod "467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" (UID: "467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab"). InnerVolumeSpecName "kube-api-access-t6w4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:07.252110 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.252078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:07.347376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.347337    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6w4w\" (UniqueName: \"kubernetes.io/projected/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-kube-api-access-t6w4w\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.347376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.347370    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:24:07.500078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.500028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:07.711490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.711447    2583 generic.go:356] "Generic (PLEG): container finished" podID="467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" containerID="93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf" exitCode=0
Sep 10 21:24:07.711721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.711513    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"
Sep 10 21:24:07.711721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.711538    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph" event={"ID":"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab","Type":"ContainerDied","Data":"93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf"}
Sep 10 21:24:07.711721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.711597    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph" event={"ID":"467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab","Type":"ContainerDied","Data":"860b2fba196b590a6f527d32236550eb169c2f2afe5974bc9d68d8bf313d92f7"}
Sep 10 21:24:07.711721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.711634    2583 scope.go:117] "RemoveContainer" containerID="93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf"
Sep 10 21:24:07.724592 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.724570    2583 scope.go:117] "RemoveContainer" containerID="93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf"
Sep 10 21:24:07.724985 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:24:07.724957    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf\": container with ID starting with 93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf not found: ID does not exist" containerID="93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf"
Sep 10 21:24:07.725117 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.724996    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf"} err="failed to get container status \"93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf\": rpc error: code = NotFound desc = could not find container \"93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf\": container with ID starting with 93cc533c9433107b6c05bd9de30eb1b1339c1966b392b93ec0c870ec245090bf not found: ID does not exist"
Sep 10 21:24:07.728986 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.728930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:07.760546 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:24:07.760435    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467f5aa8_6387_4aaa_9a12_cbebe7c2b9ab.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 21:24:07.760803 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.760776    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"]
Sep 10 21:24:07.761074 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:24:07.761051    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467f5aa8_6387_4aaa_9a12_cbebe7c2b9ab.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 21:24:07.782182 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.782149    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-780a0cdd-67bb8d6b58-2ksph"]
Sep 10 21:24:07.870595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.870553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:07.964005 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:07.963970    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" path="/var/lib/kubelet/pods/467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab/volumes"
Sep 10 21:24:08.016920 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:08.016822    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:08.249785 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:08.249749    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:08.422661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:08.422630    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:11.440589 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:11.440549    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:11.538029 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:11.537990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:11.673766 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:11.673731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:11.772394 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:11.772297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:11.942684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:11.942652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:12.196864 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:12.196828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:12.291689 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:12.291653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:20.098014 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:20.097983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:41.428760 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:41.428714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:42.311354 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:42.311314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:42.408111 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:42.408066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:42.473821 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:42.473775    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:42.567127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:42.567023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:42.621284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:42.621249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:42.713517 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:42.713468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:43.131272 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:43.131237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:43.445099 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:43.445053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:44.017807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:44.017775    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:46.469662 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:46.469621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:46.570593 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:46.570548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:47.639800 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:47.639759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:47.738413 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:47.738382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:47.855872 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:47.855837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:48.226574 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:48.226529    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:48.327979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:48.327944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:52.949045 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:24:52.949011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:03.413045 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.412939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:03.445182 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.445112    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"]
Sep 10 21:25:03.445444 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.445389    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz" podUID="dc5097ce-28f6-422c-9495-92ad3572848b" containerName="payload-processing" containerID="cri-o://f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24" gracePeriod=30
Sep 10 21:25:03.451728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.451704    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"]
Sep 10 21:25:03.452258 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.452062    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d" podUID="671be250-e7b0-4c47-aaf2-83fd141fe9ca" containerName="payload-pre-processing" containerID="cri-o://a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6" gracePeriod=30
Sep 10 21:25:03.485870 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.485817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:03.545678 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.545646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:03.622368 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.622314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:03.684917 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.684881    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:03.735407 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.735374    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:25:03.738145 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.738127    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:25:03.798055 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.798016    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5vr9\" (UniqueName: \"kubernetes.io/projected/dc5097ce-28f6-422c-9495-92ad3572848b-kube-api-access-m5vr9\") pod \"dc5097ce-28f6-422c-9495-92ad3572848b\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") "
Sep 10 21:25:03.798261 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.798116    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5097ce-28f6-422c-9495-92ad3572848b-plugins-config-volume\") pod \"dc5097ce-28f6-422c-9495-92ad3572848b\" (UID: \"dc5097ce-28f6-422c-9495-92ad3572848b\") "
Sep 10 21:25:03.798261 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.798158    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxgb4\" (UniqueName: \"kubernetes.io/projected/671be250-e7b0-4c47-aaf2-83fd141fe9ca-kube-api-access-bxgb4\") pod \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") "
Sep 10 21:25:03.798261 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.798238    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/671be250-e7b0-4c47-aaf2-83fd141fe9ca-plugins-config-volume\") pod \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\" (UID: \"671be250-e7b0-4c47-aaf2-83fd141fe9ca\") "
Sep 10 21:25:03.798933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.798898    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc5097ce-28f6-422c-9495-92ad3572848b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc5097ce-28f6-422c-9495-92ad3572848b" (UID: "dc5097ce-28f6-422c-9495-92ad3572848b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:03.798933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.798916    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/671be250-e7b0-4c47-aaf2-83fd141fe9ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "671be250-e7b0-4c47-aaf2-83fd141fe9ca" (UID: "671be250-e7b0-4c47-aaf2-83fd141fe9ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:03.799157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.799136    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc5097ce-28f6-422c-9495-92ad3572848b-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.799221 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.799164    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/671be250-e7b0-4c47-aaf2-83fd141fe9ca-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.800851 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.800824    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5097ce-28f6-422c-9495-92ad3572848b-kube-api-access-m5vr9" (OuterVolumeSpecName: "kube-api-access-m5vr9") pod "dc5097ce-28f6-422c-9495-92ad3572848b" (UID: "dc5097ce-28f6-422c-9495-92ad3572848b"). InnerVolumeSpecName "kube-api-access-m5vr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:03.801157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.801140    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/671be250-e7b0-4c47-aaf2-83fd141fe9ca-kube-api-access-bxgb4" (OuterVolumeSpecName: "kube-api-access-bxgb4") pod "671be250-e7b0-4c47-aaf2-83fd141fe9ca" (UID: "671be250-e7b0-4c47-aaf2-83fd141fe9ca"). InnerVolumeSpecName "kube-api-access-bxgb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:03.900683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.900644    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5vr9\" (UniqueName: \"kubernetes.io/projected/dc5097ce-28f6-422c-9495-92ad3572848b-kube-api-access-m5vr9\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.900683 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.900672    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxgb4\" (UniqueName: \"kubernetes.io/projected/671be250-e7b0-4c47-aaf2-83fd141fe9ca-kube-api-access-bxgb4\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:25:03.968215 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:03.968129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:04.002954 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.002921    2583 generic.go:356] "Generic (PLEG): container finished" podID="dc5097ce-28f6-422c-9495-92ad3572848b" containerID="f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24" exitCode=0
Sep 10 21:25:04.003144 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.002961    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"
Sep 10 21:25:04.003144 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.002964    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz" event={"ID":"dc5097ce-28f6-422c-9495-92ad3572848b","Type":"ContainerDied","Data":"f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24"}
Sep 10 21:25:04.003144 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.003019    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz" event={"ID":"dc5097ce-28f6-422c-9495-92ad3572848b","Type":"ContainerDied","Data":"fa3fde0021cfa2b456f988c0acad7d16be6b909b29ac191343c96b5795d3e563"}
Sep 10 21:25:04.003144 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.003042    2583 scope.go:117] "RemoveContainer" containerID="f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24"
Sep 10 21:25:04.004442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.004423    2583 generic.go:356] "Generic (PLEG): container finished" podID="671be250-e7b0-4c47-aaf2-83fd141fe9ca" containerID="a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6" exitCode=0
Sep 10 21:25:04.004575 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.004464    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"
Sep 10 21:25:04.004575 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.004517    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d" event={"ID":"671be250-e7b0-4c47-aaf2-83fd141fe9ca","Type":"ContainerDied","Data":"a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6"}
Sep 10 21:25:04.004575 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.004565    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d" event={"ID":"671be250-e7b0-4c47-aaf2-83fd141fe9ca","Type":"ContainerDied","Data":"aa9e1dbfdba0988d3ce626b16175cf89db036455df04f52231dd93b6faac2d9e"}
Sep 10 21:25:04.012065 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.012050    2583 scope.go:117] "RemoveContainer" containerID="f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24"
Sep 10 21:25:04.012367 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:25:04.012347    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24\": container with ID starting with f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24 not found: ID does not exist" containerID="f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24"
Sep 10 21:25:04.012421 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.012374    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24"} err="failed to get container status \"f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24\": rpc error: code = NotFound desc = could not find container \"f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24\": container with ID starting with f12c3a3aedea8aee066c532042636ff3682c4d97d52689222357201b0f3c7a24 not found: ID does not exist"
Sep 10 21:25:04.012421 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.012390    2583 scope.go:117] "RemoveContainer" containerID="a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6"
Sep 10 21:25:04.019748 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.019724    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"]
Sep 10 21:25:04.021866 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.021844    2583 scope.go:117] "RemoveContainer" containerID="a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6"
Sep 10 21:25:04.022163 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:25:04.022143    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6\": container with ID starting with a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6 not found: ID does not exist" containerID="a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6"
Sep 10 21:25:04.022228 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.022180    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6"} err="failed to get container status \"a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6\": rpc error: code = NotFound desc = could not find container \"a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6\": container with ID starting with a4ed3b1dfba291edb7ecee0852e5a4a556f02937ac6f755559ecba1ffad196d6 not found: ID does not exist"
Sep 10 21:25:04.022481 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.022465    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f135f4-56df487446-5kr2d"]
Sep 10 21:25:04.030883 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.030854    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"]
Sep 10 21:25:04.035287 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:04.035267    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f135f4-cffcb59b7-z5cbz"]
Sep 10 21:25:05.966121 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:05.966084    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="671be250-e7b0-4c47-aaf2-83fd141fe9ca" path="/var/lib/kubelet/pods/671be250-e7b0-4c47-aaf2-83fd141fe9ca/volumes"
Sep 10 21:25:05.966655 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:05.966634    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc5097ce-28f6-422c-9495-92ad3572848b" path="/var/lib/kubelet/pods/dc5097ce-28f6-422c-9495-92ad3572848b/volumes"
Sep 10 21:25:08.648457 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:08.648405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:09.050077 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:09.049977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:09.945386 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:09.945357    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:25:09.947432 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:09.947412    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:25:28.443140 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:28.443097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:28.544016 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:28.543965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:28.643220 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:28.643183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:28.744718 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:28.744593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:29.743698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:29.743664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:30.151664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:30.151617    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:30.454864 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:30.454777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:30.679641 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:30.679610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:32.677334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:32.677297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:32.777804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:32.777768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:33.606622 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:33.606586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:33.885387 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:33.885299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:33.990554 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:33.990519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:34.393040 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:34.392998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:34.488286 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:34.488254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:34.588612 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:34.588571    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:38.720612 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:38.720579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:46.010605 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:46.010572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:46.239092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:46.239052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:46.686229 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:46.686192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:46.997814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:46.997719    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:47.086537 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:47.086480    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:50.311908 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:50.311872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:50.411479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:50.411441    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:51.577333 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.577299    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"]
Sep 10 21:25:51.577942 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.577916    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671be250-e7b0-4c47-aaf2-83fd141fe9ca" containerName="payload-pre-processing"
Sep 10 21:25:51.578039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.577944    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="671be250-e7b0-4c47-aaf2-83fd141fe9ca" containerName="payload-pre-processing"
Sep 10 21:25:51.578039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.577975    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" containerName="payload-pre-processing"
Sep 10 21:25:51.578039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.577983    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" containerName="payload-pre-processing"
Sep 10 21:25:51.578039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.578003    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc5097ce-28f6-422c-9495-92ad3572848b" containerName="payload-processing"
Sep 10 21:25:51.578039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.578012    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc5097ce-28f6-422c-9495-92ad3572848b" containerName="payload-processing"
Sep 10 21:25:51.578301 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.578124    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="671be250-e7b0-4c47-aaf2-83fd141fe9ca" containerName="payload-pre-processing"
Sep 10 21:25:51.578301 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.578137    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc5097ce-28f6-422c-9495-92ad3572848b" containerName="payload-processing"
Sep 10 21:25:51.578301 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.578148    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="467f5aa8-6387-4aaa-9a12-cbebe7c2b9ab" containerName="payload-pre-processing"
Sep 10 21:25:51.581764 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.581744    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.583858 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.583836    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-fbe4cf-serving-cert\""
Sep 10 21:25:51.593904 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.593598    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"]
Sep 10 21:25:51.609776 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.609724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:51.666906 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.666874    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-maas-api-tls\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.667112 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.666990    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-openshift-service-ca\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.667112 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.667027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-metrics-certs\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.667112 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.667059    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s54mb\" (UniqueName: \"kubernetes.io/projected/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-kube-api-access-s54mb\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.669971 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.669927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:51.768442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.768408    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-openshift-service-ca\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.768442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.768446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-metrics-certs\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.768696 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.768472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s54mb\" (UniqueName: \"kubernetes.io/projected/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-kube-api-access-s54mb\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.768696 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.768542    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-maas-api-tls\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.769164 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.769139    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-openshift-service-ca\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.770954 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.770930    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-maas-api-tls\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.771063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.770984    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-metrics-certs\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.775934 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.775913    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s54mb\" (UniqueName: \"kubernetes.io/projected/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-kube-api-access-s54mb\") pod \"maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:51.777399 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.777378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:51.898813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:51.898782    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:52.115080 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.115045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.177567 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.177543    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"]
Sep 10 21:25:52.180208 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:25:52.180178    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b70fec5_74d9_4d9e_a4ac_de95f012a2e3.slice/crio-7571a2b0e6127c2dce6b04b38be11bbce1a4da152c48676f39958e8a8ff17b1d WatchSource:0}: Error finding container 7571a2b0e6127c2dce6b04b38be11bbce1a4da152c48676f39958e8a8ff17b1d: Status 404 returned error can't find the container with id 7571a2b0e6127c2dce6b04b38be11bbce1a4da152c48676f39958e8a8ff17b1d
Sep 10 21:25:52.210056 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.210005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.222874 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.222843    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"]
Sep 10 21:25:52.227402 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.227385    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.229513 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.229476    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-fbe4cf-dockercfg-b2bgd\""
Sep 10 21:25:52.229635 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.229546    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-fbe4cf\""
Sep 10 21:25:52.236917 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.236883    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"]
Sep 10 21:25:52.256718 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.256648    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm" event={"ID":"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3","Type":"ContainerStarted","Data":"7571a2b0e6127c2dce6b04b38be11bbce1a4da152c48676f39958e8a8ff17b1d"}
Sep 10 21:25:52.273166 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.273111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69sx2\" (UniqueName: \"kubernetes.io/projected/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-kube-api-access-69sx2\") pod \"payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.273382 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.273251    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.374716 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.374672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.374889 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.374794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69sx2\" (UniqueName: \"kubernetes.io/projected/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-kube-api-access-69sx2\") pod \"payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.375375 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.375357    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.384798 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.384767    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69sx2\" (UniqueName: \"kubernetes.io/projected/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-kube-api-access-69sx2\") pod \"payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.539979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.539872    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:52.812804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:52.812711    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"]
Sep 10 21:25:52.816251 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:25:52.816208    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9e059a_58dc_4d2c_96fb_4b371fa9c388.slice/crio-118b7982a851e6e91a65e78b4c8aa8149a9e9ec26d685a11acce2267c0dca5f9 WatchSource:0}: Error finding container 118b7982a851e6e91a65e78b4c8aa8149a9e9ec26d685a11acce2267c0dca5f9: Status 404 returned error can't find the container with id 118b7982a851e6e91a65e78b4c8aa8149a9e9ec26d685a11acce2267c0dca5f9
Sep 10 21:25:53.263463 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:53.263423    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm" event={"ID":"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3","Type":"ContainerStarted","Data":"917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1"}
Sep 10 21:25:53.263698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:53.263535    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:53.265012 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:53.264990    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67" event={"ID":"0c9e059a-58dc-4d2c-96fb-4b371fa9c388","Type":"ContainerStarted","Data":"a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4"}
Sep 10 21:25:53.265135 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:53.265029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67" event={"ID":"0c9e059a-58dc-4d2c-96fb-4b371fa9c388","Type":"ContainerStarted","Data":"118b7982a851e6e91a65e78b4c8aa8149a9e9ec26d685a11acce2267c0dca5f9"}
Sep 10 21:25:53.265135 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:53.265048    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:25:53.280106 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:53.280046    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm" podStartSLOduration=1.872027708 podStartE2EDuration="2.280029631s" totalImagesPullingTime="408.001923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:53.277922904 +0000 UTC m=+943.804887250" watchObservedRunningTime="2026-09-10 21:25:53.280029631 +0000 UTC m=+943.806993978"
Sep 10 21:25:54.800670 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:54.800632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:59.276267 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:59.276239    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:25:59.293895 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:25:59.293848    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67" podStartSLOduration=7.293832524 podStartE2EDuration="7.293832524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:53.294045211 +0000 UTC m=+943.821009560" watchObservedRunningTime="2026-09-10 21:25:59.293832524 +0000 UTC m=+949.820796904"
Sep 10 21:26:00.396747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:00.396713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:04.272692 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:04.272662    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:26:04.598642 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:04.598544    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:04.678664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:04.678628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:04.747773 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:04.747712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:04.822958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:04.822928    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:04.884824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:04.884715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:04.982366 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:04.982327    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:05.595420 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:05.595379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:05.692681 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:05.692650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:05.792877 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:05.792845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:06.806039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:06.805955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:07.178540 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:07.178475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.107523 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:13.107471    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.172697 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:13.172665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.253971 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:13.253923    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.354908 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:13.354861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.455094 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:13.455059    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.607042 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:13.607013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:13.782918 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:13.782829    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:15.227528 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:15.227472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:17.610948 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:17.610908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:25.537316 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:25.537282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:36.019112 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:36.019028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:39.684707 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:39.684671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:40.307306 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:40.307266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:43.347294 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:43.347261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:44.464251 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:44.464215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:45.656452 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:45.656413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:45.889862 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:45.889826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:45.999148 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:45.999054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:46.089900 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:46.089859    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:55.142063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.142024    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"]
Sep 10 21:26:55.142473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.142232    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm" podUID="2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" containerName="maas-api" containerID="cri-o://917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1" gracePeriod=30
Sep 10 21:26:55.171004 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.170965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:55.208568 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.208526    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"]
Sep 10 21:26:55.208785 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.208759    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67" podUID="0c9e059a-58dc-4d2c-96fb-4b371fa9c388" containerName="payload-pre-processing" containerID="cri-o://a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4" gracePeriod=30
Sep 10 21:26:55.271063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.271026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:55.374718 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.374678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:55.447224 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.447183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:55.466662 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.466638    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:26:55.500780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.499384    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-metrics-certs\") pod \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") "
Sep 10 21:26:55.500780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.499452    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-maas-api-tls\") pod \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") "
Sep 10 21:26:55.500780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.499489    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s54mb\" (UniqueName: \"kubernetes.io/projected/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-kube-api-access-s54mb\") pod \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") "
Sep 10 21:26:55.500780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.499652    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-openshift-service-ca\") pod \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\" (UID: \"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3\") "
Sep 10 21:26:55.500780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.500319    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" (UID: "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:55.504065 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.504029    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-kube-api-access-s54mb" (OuterVolumeSpecName: "kube-api-access-s54mb") pod "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" (UID: "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3"). InnerVolumeSpecName "kube-api-access-s54mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:55.506565 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.506485    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" (UID: "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:55.508922 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.508897    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" (UID: "2b70fec5-74d9-4d9e-a4ac-de95f012a2e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:26:55.514265 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.514241    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:26:55.587579 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.587541    2583 generic.go:356] "Generic (PLEG): container finished" podID="0c9e059a-58dc-4d2c-96fb-4b371fa9c388" containerID="a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4" exitCode=0
Sep 10 21:26:55.587765 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.587603    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"
Sep 10 21:26:55.587765 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.587614    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67" event={"ID":"0c9e059a-58dc-4d2c-96fb-4b371fa9c388","Type":"ContainerDied","Data":"a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4"}
Sep 10 21:26:55.587765 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.587703    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67" event={"ID":"0c9e059a-58dc-4d2c-96fb-4b371fa9c388","Type":"ContainerDied","Data":"118b7982a851e6e91a65e78b4c8aa8149a9e9ec26d685a11acce2267c0dca5f9"}
Sep 10 21:26:55.587765 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.587728    2583 scope.go:117] "RemoveContainer" containerID="a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4"
Sep 10 21:26:55.589022 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.589000    2583 generic.go:356] "Generic (PLEG): container finished" podID="2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" containerID="917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1" exitCode=0
Sep 10 21:26:55.589125 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.589042    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm" event={"ID":"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3","Type":"ContainerDied","Data":"917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1"}
Sep 10 21:26:55.589125 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.589057    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"
Sep 10 21:26:55.589125 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.589076    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm" event={"ID":"2b70fec5-74d9-4d9e-a4ac-de95f012a2e3","Type":"ContainerDied","Data":"7571a2b0e6127c2dce6b04b38be11bbce1a4da152c48676f39958e8a8ff17b1d"}
Sep 10 21:26:55.598494 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.598476    2583 scope.go:117] "RemoveContainer" containerID="a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4"
Sep 10 21:26:55.598827 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:26:55.598806    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4\": container with ID starting with a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4 not found: ID does not exist" containerID="a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4"
Sep 10 21:26:55.598904 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.598835    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4"} err="failed to get container status \"a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4\": rpc error: code = NotFound desc = could not find container \"a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4\": container with ID starting with a480200a27e0b66c6098f7021d1cdaff48514918fa5dc939a87c5a8cfbdeafb4 not found: ID does not exist"
Sep 10 21:26:55.598904 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.598851    2583 scope.go:117] "RemoveContainer" containerID="917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1"
Sep 10 21:26:55.601120 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601097    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-plugins-config-volume\") pod \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") "
Sep 10 21:26:55.601236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601220    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69sx2\" (UniqueName: \"kubernetes.io/projected/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-kube-api-access-69sx2\") pod \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\" (UID: \"0c9e059a-58dc-4d2c-96fb-4b371fa9c388\") "
Sep 10 21:26:55.601430 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601409    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c9e059a-58dc-4d2c-96fb-4b371fa9c388" (UID: "0c9e059a-58dc-4d2c-96fb-4b371fa9c388"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:55.601556 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601540    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-metrics-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.601556 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601555    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-maas-api-tls\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.601664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601564    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.601664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601573    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s54mb\" (UniqueName: \"kubernetes.io/projected/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-kube-api-access-s54mb\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.601664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.601581    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3-openshift-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.603341 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.603319    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-kube-api-access-69sx2" (OuterVolumeSpecName: "kube-api-access-69sx2") pod "0c9e059a-58dc-4d2c-96fb-4b371fa9c388" (UID: "0c9e059a-58dc-4d2c-96fb-4b371fa9c388"). InnerVolumeSpecName "kube-api-access-69sx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:55.607725 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.607706    2583 scope.go:117] "RemoveContainer" containerID="917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1"
Sep 10 21:26:55.607993 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:26:55.607976    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1\": container with ID starting with 917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1 not found: ID does not exist" containerID="917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1"
Sep 10 21:26:55.608046 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.608000    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1"} err="failed to get container status \"917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1\": rpc error: code = NotFound desc = could not find container \"917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1\": container with ID starting with 917b6027de3640f1501f49458d61b13bc98caec02771f4d50157e44b528f51c1 not found: ID does not exist"
Sep 10 21:26:55.612140 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.612117    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"]
Sep 10 21:26:55.616739 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.616719    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-fbe4cf-86d4f8d765-zkbqm"]
Sep 10 21:26:55.702322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.702234    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69sx2\" (UniqueName: \"kubernetes.io/projected/0c9e059a-58dc-4d2c-96fb-4b371fa9c388-kube-api-access-69sx2\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:26:55.767410 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.767373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:55.910664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.910635    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"]
Sep 10 21:26:55.915615 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.915589    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-fbe4cf-57d6c5865d-rnh67"]
Sep 10 21:26:55.961529 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.961434    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c9e059a-58dc-4d2c-96fb-4b371fa9c388" path="/var/lib/kubelet/pods/0c9e059a-58dc-4d2c-96fb-4b371fa9c388/volumes"
Sep 10 21:26:55.961906 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:55.961891    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" path="/var/lib/kubelet/pods/2b70fec5-74d9-4d9e-a4ac-de95f012a2e3/volumes"
Sep 10 21:26:59.281791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:26:59.281756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:01.991174 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:01.991141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:02.217056 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:02.217023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:02.317762 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:02.317665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:02.486189 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:02.486151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:02.943668 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:02.943625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:03.227806 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:03.227700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:03.662601 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:03.662567    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:04.216412 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:04.216374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:04.317399 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:04.317363    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:04.418564 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:04.418525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:07.871268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:07.871229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:07.939738 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:07.939706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:08.025559 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:08.025517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:08.127702 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:08.127589    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:17.744601 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:17.744566    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:18.521146 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:18.521105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:19.295707 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:19.295672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:25.214007 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:25.213973    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:25.315324 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:25.315288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:25.412296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:25.412263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:25.514470 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:25.514383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:26.115974 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:26.115943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:26.216252 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:26.216220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:26.326879 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:26.326844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:26.986007 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:26.985973    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:27.102950 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:27.102917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:27.321144 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:27.321039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:27.425199 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:27.425162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:27.771715 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:27.771681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:32.435330 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:32.435287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:33.518466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:33.518429    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:33.618459 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:33.618419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:34.180397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:34.180353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:40.376495 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:40.376462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:40.476855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:40.476815    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:40.591992 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:40.591956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:40.916909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:40.916872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:41.283202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:41.283092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.136379 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:44.136323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.220366 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:44.220327    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.334701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:45.334656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.418140 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:45.418100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.513172 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:45.513136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.967195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:45.967159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:46.064051 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.064017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:46.149109 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.149075    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"]
Sep 10 21:27:46.149465 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.149453    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" containerName="maas-api"
Sep 10 21:27:46.149532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.149467    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" containerName="maas-api"
Sep 10 21:27:46.149532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.149494    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9e059a-58dc-4d2c-96fb-4b371fa9c388" containerName="payload-pre-processing"
Sep 10 21:27:46.149532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.149512    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9e059a-58dc-4d2c-96fb-4b371fa9c388" containerName="payload-pre-processing"
Sep 10 21:27:46.149631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.149568    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b70fec5-74d9-4d9e-a4ac-de95f012a2e3" containerName="maas-api"
Sep 10 21:27:46.149631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.149577    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9e059a-58dc-4d2c-96fb-4b371fa9c388" containerName="payload-pre-processing"
Sep 10 21:27:46.152829 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.152813    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.154700 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.154673    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-76320447\""
Sep 10 21:27:46.154826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.154744    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-76320447-dockercfg-wdx5x\""
Sep 10 21:27:46.162938 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.162912    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"]
Sep 10 21:27:46.193561 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.193529    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rdbc\" (UniqueName: \"kubernetes.io/projected/08a4c5ac-a18f-457a-bcf1-fef288250fd4-kube-api-access-5rdbc\") pod \"payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.193746 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.193684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08a4c5ac-a18f-457a-bcf1-fef288250fd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.294370 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.294281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rdbc\" (UniqueName: \"kubernetes.io/projected/08a4c5ac-a18f-457a-bcf1-fef288250fd4-kube-api-access-5rdbc\") pod \"payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.294370 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.294363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08a4c5ac-a18f-457a-bcf1-fef288250fd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.295090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.295069    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08a4c5ac-a18f-457a-bcf1-fef288250fd4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.301875 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.301844    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rdbc\" (UniqueName: \"kubernetes.io/projected/08a4c5ac-a18f-457a-bcf1-fef288250fd4-kube-api-access-5rdbc\") pod \"payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.465133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.465088    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.767340 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.767311    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"]
Sep 10 21:27:46.769529 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:27:46.769481    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a4c5ac_a18f_457a_bcf1_fef288250fd4.slice/crio-81c2732892197e99bad82942b8e0c7e3b33df31ebcf3c22de14275a35f86d486 WatchSource:0}: Error finding container 81c2732892197e99bad82942b8e0c7e3b33df31ebcf3c22de14275a35f86d486: Status 404 returned error can't find the container with id 81c2732892197e99bad82942b8e0c7e3b33df31ebcf3c22de14275a35f86d486
Sep 10 21:27:46.867112 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.867078    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7" event={"ID":"08a4c5ac-a18f-457a-bcf1-fef288250fd4","Type":"ContainerStarted","Data":"fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba"}
Sep 10 21:27:46.867334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.867127    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7" event={"ID":"08a4c5ac-a18f-457a-bcf1-fef288250fd4","Type":"ContainerStarted","Data":"81c2732892197e99bad82942b8e0c7e3b33df31ebcf3c22de14275a35f86d486"}
Sep 10 21:27:46.867334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.867179    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:27:46.884141 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:46.884086    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7" podStartSLOduration=0.884068237 podStartE2EDuration="884.068237ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:27:46.88216029 +0000 UTC m=+1057.409124650" watchObservedRunningTime="2026-09-10 21:27:46.884068237 +0000 UTC m=+1057.411032584"
Sep 10 21:27:49.281285 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:49.281253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:50.157969 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:50.157927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:50.511217 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:50.511123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:57.874576 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:27:57.874540    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:28:00.909379 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:00.909344    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"]
Sep 10 21:28:00.909994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:00.909615    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n" podUID="b8173489-ac09-4b21-a110-866bd9844723" containerName="maas-api" containerID="cri-o://6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a" gracePeriod=30
Sep 10 21:28:00.941316 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:00.941282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:00.978082 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:00.978044    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"]
Sep 10 21:28:00.978453 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:00.978329    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r" podUID="67ca510d-5611-4d2b-859e-497df0cfb0f8" containerName="payload-processing" containerID="cri-o://d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214" gracePeriod=30
Sep 10 21:28:00.986881 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:00.986848    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"]
Sep 10 21:28:00.987408 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:00.987083    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn" podUID="e5ae1169-2e9e-4af6-be49-e61222866bd7" containerName="payload-pre-processing" containerID="cri-o://b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d" gracePeriod=30
Sep 10 21:28:01.029227 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.029193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:01.127920 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.127871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:01.193141 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.193106    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:28:01.199544 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.199516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:01.263924 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.263894    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:28:01.291167 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.291145    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:28:01.342294 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.342252    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls\") pod \"b8173489-ac09-4b21-a110-866bd9844723\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") "
Sep 10 21:28:01.342294 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.342298    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-metrics-certs\") pod \"b8173489-ac09-4b21-a110-866bd9844723\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") "
Sep 10 21:28:01.342571 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.342421    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67ca510d-5611-4d2b-859e-497df0cfb0f8-plugins-config-volume\") pod \"67ca510d-5611-4d2b-859e-497df0cfb0f8\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") "
Sep 10 21:28:01.342571 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.342467    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8173489-ac09-4b21-a110-866bd9844723-openshift-service-ca\") pod \"b8173489-ac09-4b21-a110-866bd9844723\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") "
Sep 10 21:28:01.342571 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.342534    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trklh\" (UniqueName: \"kubernetes.io/projected/67ca510d-5611-4d2b-859e-497df0cfb0f8-kube-api-access-trklh\") pod \"67ca510d-5611-4d2b-859e-497df0cfb0f8\" (UID: \"67ca510d-5611-4d2b-859e-497df0cfb0f8\") "
Sep 10 21:28:01.342571 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.342564    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmd6x\" (UniqueName: \"kubernetes.io/projected/b8173489-ac09-4b21-a110-866bd9844723-kube-api-access-pmd6x\") pod \"b8173489-ac09-4b21-a110-866bd9844723\" (UID: \"b8173489-ac09-4b21-a110-866bd9844723\") "
Sep 10 21:28:01.343279 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.343228    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ca510d-5611-4d2b-859e-497df0cfb0f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67ca510d-5611-4d2b-859e-497df0cfb0f8" (UID: "67ca510d-5611-4d2b-859e-497df0cfb0f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:01.345191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.344156    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8173489-ac09-4b21-a110-866bd9844723-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8173489-ac09-4b21-a110-866bd9844723" (UID: "b8173489-ac09-4b21-a110-866bd9844723"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:01.349432 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.349385    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b8173489-ac09-4b21-a110-866bd9844723" (UID: "b8173489-ac09-4b21-a110-866bd9844723"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:01.349955 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.349870    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8173489-ac09-4b21-a110-866bd9844723" (UID: "b8173489-ac09-4b21-a110-866bd9844723"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:01.349955 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.349891    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8173489-ac09-4b21-a110-866bd9844723-kube-api-access-pmd6x" (OuterVolumeSpecName: "kube-api-access-pmd6x") pod "b8173489-ac09-4b21-a110-866bd9844723" (UID: "b8173489-ac09-4b21-a110-866bd9844723"). InnerVolumeSpecName "kube-api-access-pmd6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:01.349955 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.349925    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ca510d-5611-4d2b-859e-497df0cfb0f8-kube-api-access-trklh" (OuterVolumeSpecName: "kube-api-access-trklh") pod "67ca510d-5611-4d2b-859e-497df0cfb0f8" (UID: "67ca510d-5611-4d2b-859e-497df0cfb0f8"). InnerVolumeSpecName "kube-api-access-trklh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:01.443652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443568    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68rrp\" (UniqueName: \"kubernetes.io/projected/e5ae1169-2e9e-4af6-be49-e61222866bd7-kube-api-access-68rrp\") pod \"e5ae1169-2e9e-4af6-be49-e61222866bd7\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") "
Sep 10 21:28:01.443652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443616    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ae1169-2e9e-4af6-be49-e61222866bd7-plugins-config-volume\") pod \"e5ae1169-2e9e-4af6-be49-e61222866bd7\" (UID: \"e5ae1169-2e9e-4af6-be49-e61222866bd7\") "
Sep 10 21:28:01.443841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443802    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trklh\" (UniqueName: \"kubernetes.io/projected/67ca510d-5611-4d2b-859e-497df0cfb0f8-kube-api-access-trklh\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.443841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443815    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmd6x\" (UniqueName: \"kubernetes.io/projected/b8173489-ac09-4b21-a110-866bd9844723-kube-api-access-pmd6x\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.443841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443826    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-maas-api-tls\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.443841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443835    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8173489-ac09-4b21-a110-866bd9844723-metrics-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.443970 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443844    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67ca510d-5611-4d2b-859e-497df0cfb0f8-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.443970 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443853    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8173489-ac09-4b21-a110-866bd9844723-openshift-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.444036 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.443975    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ae1169-2e9e-4af6-be49-e61222866bd7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5ae1169-2e9e-4af6-be49-e61222866bd7" (UID: "e5ae1169-2e9e-4af6-be49-e61222866bd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:01.445682 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.445662    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ae1169-2e9e-4af6-be49-e61222866bd7-kube-api-access-68rrp" (OuterVolumeSpecName: "kube-api-access-68rrp") pod "e5ae1169-2e9e-4af6-be49-e61222866bd7" (UID: "e5ae1169-2e9e-4af6-be49-e61222866bd7"). InnerVolumeSpecName "kube-api-access-68rrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:01.536165 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.536128    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:01.545300 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.545272    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68rrp\" (UniqueName: \"kubernetes.io/projected/e5ae1169-2e9e-4af6-be49-e61222866bd7-kube-api-access-68rrp\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.545385 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.545303    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5ae1169-2e9e-4af6-be49-e61222866bd7-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:01.946030 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.945986    2583 generic.go:356] "Generic (PLEG): container finished" podID="e5ae1169-2e9e-4af6-be49-e61222866bd7" containerID="b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d" exitCode=0
Sep 10 21:28:01.946462 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.946034    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"
Sep 10 21:28:01.946462 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.946065    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn" event={"ID":"e5ae1169-2e9e-4af6-be49-e61222866bd7","Type":"ContainerDied","Data":"b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d"}
Sep 10 21:28:01.946462 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.946150    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn" event={"ID":"e5ae1169-2e9e-4af6-be49-e61222866bd7","Type":"ContainerDied","Data":"1e74dba1cbd72eba51de9165ba5ed9feb4bf65245590c5a2ec6b0e6137a46029"}
Sep 10 21:28:01.946462 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.946184    2583 scope.go:117] "RemoveContainer" containerID="b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d"
Sep 10 21:28:01.947627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.947556    2583 generic.go:356] "Generic (PLEG): container finished" podID="b8173489-ac09-4b21-a110-866bd9844723" containerID="6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a" exitCode=0
Sep 10 21:28:01.947627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.947603    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"
Sep 10 21:28:01.947756 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.947637    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n" event={"ID":"b8173489-ac09-4b21-a110-866bd9844723","Type":"ContainerDied","Data":"6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a"}
Sep 10 21:28:01.947756 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.947680    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n" event={"ID":"b8173489-ac09-4b21-a110-866bd9844723","Type":"ContainerDied","Data":"6bf2a2e8bebcb4d4707f03a938a9cb1d42ba2c5d66e9e7ebf79eed5790a63e8d"}
Sep 10 21:28:01.949235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.949210    2583 generic.go:356] "Generic (PLEG): container finished" podID="67ca510d-5611-4d2b-859e-497df0cfb0f8" containerID="d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214" exitCode=0
Sep 10 21:28:01.949361 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.949274    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"
Sep 10 21:28:01.949361 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.949294    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r" event={"ID":"67ca510d-5611-4d2b-859e-497df0cfb0f8","Type":"ContainerDied","Data":"d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214"}
Sep 10 21:28:01.949480 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.949371    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r" event={"ID":"67ca510d-5611-4d2b-859e-497df0cfb0f8","Type":"ContainerDied","Data":"0a6fdab376874ab0a811f86c561ce1b882e205d26c8362fef3b5c4db67b42e5d"}
Sep 10 21:28:01.956579 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.956361    2583 scope.go:117] "RemoveContainer" containerID="b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d"
Sep 10 21:28:01.956756 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:01.956728    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d\": container with ID starting with b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d not found: ID does not exist" containerID="b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d"
Sep 10 21:28:01.956827 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.956767    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d"} err="failed to get container status \"b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d\": rpc error: code = NotFound desc = could not find container \"b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d\": container with ID starting with b6d30e541f3721632adf62d6fcf86d12f8199f57b3ec6b8764d52d2cb9759a2d not found: ID does not exist"
Sep 10 21:28:01.956827 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.956789    2583 scope.go:117] "RemoveContainer" containerID="6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a"
Sep 10 21:28:01.966850 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.966826    2583 scope.go:117] "RemoveContainer" containerID="6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a"
Sep 10 21:28:01.967184 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:01.967156    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a\": container with ID starting with 6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a not found: ID does not exist" containerID="6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a"
Sep 10 21:28:01.967280 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.967194    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a"} err="failed to get container status \"6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a\": rpc error: code = NotFound desc = could not find container \"6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a\": container with ID starting with 6eb0c032300c74cdc67ce873a516146cdd0695feb3effccb34244f82c28b9d5a not found: ID does not exist"
Sep 10 21:28:01.967280 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.967216    2583 scope.go:117] "RemoveContainer" containerID="d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214"
Sep 10 21:28:01.975383 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.975360    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"]
Sep 10 21:28:01.975706 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.975691    2583 scope.go:117] "RemoveContainer" containerID="d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214"
Sep 10 21:28:01.975968 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:01.975949    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214\": container with ID starting with d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214 not found: ID does not exist" containerID="d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214"
Sep 10 21:28:01.976013 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.975976    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214"} err="failed to get container status \"d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214\": rpc error: code = NotFound desc = could not find container \"d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214\": container with ID starting with d49cb136af12b5248e1d81d68e0aa027379836c5a007c4be1471781910298214 not found: ID does not exist"
Sep 10 21:28:01.982724 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.982702    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ec041-656477bc7d-fmpzn"]
Sep 10 21:28:01.992528 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.992491    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"]
Sep 10 21:28:01.996218 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:01.996187    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5ec041-c7875ccb6-9gp9n"]
Sep 10 21:28:02.004926 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:02.004901    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"]
Sep 10 21:28:02.009335 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:02.009315    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ec041-78c748549-pkm8r"]
Sep 10 21:28:03.961255 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:03.961225    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67ca510d-5611-4d2b-859e-497df0cfb0f8" path="/var/lib/kubelet/pods/67ca510d-5611-4d2b-859e-497df0cfb0f8/volumes"
Sep 10 21:28:03.961677 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:03.961663    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8173489-ac09-4b21-a110-866bd9844723" path="/var/lib/kubelet/pods/b8173489-ac09-4b21-a110-866bd9844723/volumes"
Sep 10 21:28:03.961979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:03.961968    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ae1169-2e9e-4af6-be49-e61222866bd7" path="/var/lib/kubelet/pods/e5ae1169-2e9e-4af6-be49-e61222866bd7/volumes"
Sep 10 21:28:06.473280 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:06.473248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:07.907721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:07.907684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:07.977900 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:07.977863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:08.153219 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:08.153184    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"]
Sep 10 21:28:08.153539 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:08.153489    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy" containerID="cri-o://2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea" gracePeriod=30
Sep 10 21:28:08.176601 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:08.176486    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:08.606435 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:08.606356    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:09.064707 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:09.064671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:09.677097 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:09.677061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:09.773757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:09.773721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:10.079345 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:10.079306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:10.457979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:10.457940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:13.185243 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:13.185216    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c074bb_314e_43e4_80d9_0fbdb8eef015.slice/crio-conmon-2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:28:13.420791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.420713    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:28:13.559712 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559683    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-socket\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.559892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559723    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-certs\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.559892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559747    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c074bb-314e-43e4-80d9-0fbdb8eef015-istiod-ca-cert\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.559892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559797    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-data\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.559892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559825    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnjlp\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-kube-api-access-fnjlp\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.559892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559887    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-podinfo\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.560090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559910    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-credential-socket\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.560090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559967    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-envoy\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.560090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559970    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:13.560090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.559990    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:13.560090 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560007    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-token\") pod \"32c074bb-314e-43e4-80d9-0fbdb8eef015\" (UID: \"32c074bb-314e-43e4-80d9-0fbdb8eef015\") "
Sep 10 21:28:13.560298 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560105    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-data" (OuterVolumeSpecName: "istio-data") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:13.560298 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560235    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:13.560298 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560281    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c074bb-314e-43e4-80d9-0fbdb8eef015-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:13.560443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560400    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.560443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560417    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-workload-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.560443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560426    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32c074bb-314e-43e4-80d9-0fbdb8eef015-istiod-ca-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.560443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560434    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-data\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.560443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.560442    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-credential-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.562267 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.562235    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-token" (OuterVolumeSpecName: "istio-token") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:13.562267 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.562253    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:28:13.562539 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.562513    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:28:13.562655 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.562514    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-kube-api-access-fnjlp" (OuterVolumeSpecName: "kube-api-access-fnjlp") pod "32c074bb-314e-43e4-80d9-0fbdb8eef015" (UID: "32c074bb-314e-43e4-80d9-0fbdb8eef015"). InnerVolumeSpecName "kube-api-access-fnjlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:13.661855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.661817    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-token\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.661855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.661846    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnjlp\" (UniqueName: \"kubernetes.io/projected/32c074bb-314e-43e4-80d9-0fbdb8eef015-kube-api-access-fnjlp\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.661855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.661859    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-podinfo\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.662094 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:13.661867    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32c074bb-314e-43e4-80d9-0fbdb8eef015-istio-envoy\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:14.012468 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.012435    2583 generic.go:356] "Generic (PLEG): container finished" podID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerID="2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea" exitCode=0
Sep 10 21:28:14.012661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.012519    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"
Sep 10 21:28:14.012661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.012538    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" event={"ID":"32c074bb-314e-43e4-80d9-0fbdb8eef015","Type":"ContainerDied","Data":"2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea"}
Sep 10 21:28:14.012661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.012607    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp" event={"ID":"32c074bb-314e-43e4-80d9-0fbdb8eef015","Type":"ContainerDied","Data":"cdc69c83148c6adbab833ee6e3e18a5a1eafb5d267e1118ae024b28da1da3255"}
Sep 10 21:28:14.012661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.012638    2583 scope.go:117] "RemoveContainer" containerID="2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea"
Sep 10 21:28:14.021548 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.021531    2583 scope.go:117] "RemoveContainer" containerID="495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4"
Sep 10 21:28:14.030690 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.030661    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"]
Sep 10 21:28:14.030865 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.030813    2583 scope.go:117] "RemoveContainer" containerID="2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea"
Sep 10 21:28:14.031202 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:14.031173    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea\": container with ID starting with 2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea not found: ID does not exist" containerID="2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea"
Sep 10 21:28:14.031312 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.031212    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea"} err="failed to get container status \"2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea\": rpc error: code = NotFound desc = could not find container \"2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea\": container with ID starting with 2f9762562d51bd94fbbab793ba9f99a48348c180735545272a8fdb15bbb077ea not found: ID does not exist"
Sep 10 21:28:14.031312 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.031233    2583 scope.go:117] "RemoveContainer" containerID="495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4"
Sep 10 21:28:14.031884 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:14.031861    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4\": container with ID starting with 495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4 not found: ID does not exist" containerID="495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4"
Sep 10 21:28:14.031978 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.031895    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4"} err="failed to get container status \"495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4\": rpc error: code = NotFound desc = could not find container \"495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4\": container with ID starting with 495cb978809c74b1b699882aae5bf72560459247ee4d7c8b0077b49eb67218c4 not found: ID does not exist"
Sep 10 21:28:14.033279 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:14.033259    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-5ec041-openshift-default-d5d79b966-qp4jp"]
Sep 10 21:28:15.870903 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:15.870868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:15.961388 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:15.961355    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" path="/var/lib/kubelet/pods/32c074bb-314e-43e4-80d9-0fbdb8eef015/volumes"
Sep 10 21:28:15.971210 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:15.971164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:16.236465 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236434    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"]
Sep 10 21:28:16.236847 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236831    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy"
Sep 10 21:28:16.236891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236849    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy"
Sep 10 21:28:16.236891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236860    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy"
Sep 10 21:28:16.236891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236865    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy"
Sep 10 21:28:16.236891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236872    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8173489-ac09-4b21-a110-866bd9844723" containerName="maas-api"
Sep 10 21:28:16.236891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236878    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8173489-ac09-4b21-a110-866bd9844723" containerName="maas-api"
Sep 10 21:28:16.236891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236890    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ae1169-2e9e-4af6-be49-e61222866bd7" containerName="payload-pre-processing"
Sep 10 21:28:16.237071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236895    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ae1169-2e9e-4af6-be49-e61222866bd7" containerName="payload-pre-processing"
Sep 10 21:28:16.237071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236910    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ca510d-5611-4d2b-859e-497df0cfb0f8" containerName="payload-processing"
Sep 10 21:28:16.237071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236916    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ca510d-5611-4d2b-859e-497df0cfb0f8" containerName="payload-processing"
Sep 10 21:28:16.237071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236970    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy"
Sep 10 21:28:16.237071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236980    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8173489-ac09-4b21-a110-866bd9844723" containerName="maas-api"
Sep 10 21:28:16.237071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236986    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ae1169-2e9e-4af6-be49-e61222866bd7" containerName="payload-pre-processing"
Sep 10 21:28:16.237071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.236992    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="67ca510d-5611-4d2b-859e-497df0cfb0f8" containerName="payload-processing"
Sep 10 21:28:16.241590 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.241564    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"
Sep 10 21:28:16.243786 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.243763    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:28:16.248452 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.248420    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"]
Sep 10 21:28:16.283852 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.283820    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c7a520-60f8-484f-8dee-2bc726a38d13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-76320447-2xhwd\" (UID: \"69c7a520-60f8-484f-8dee-2bc726a38d13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"
Sep 10 21:28:16.384686 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.384643    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c7a520-60f8-484f-8dee-2bc726a38d13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-76320447-2xhwd\" (UID: \"69c7a520-60f8-484f-8dee-2bc726a38d13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"
Sep 10 21:28:16.385617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.385594    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c7a520-60f8-484f-8dee-2bc726a38d13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-76320447-2xhwd\" (UID: \"69c7a520-60f8-484f-8dee-2bc726a38d13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"
Sep 10 21:28:16.387846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.387818    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:16.555236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.555129    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"
Sep 10 21:28:16.855880 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.855812    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"]
Sep 10 21:28:16.858309 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:28:16.858279    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c7a520_60f8_484f_8dee_2bc726a38d13.slice/crio-df2cc14a819a2b3f9269030f95cbbc042868f2092cd1bdb35988ed6a6329ae26 WatchSource:0}: Error finding container df2cc14a819a2b3f9269030f95cbbc042868f2092cd1bdb35988ed6a6329ae26: Status 404 returned error can't find the container with id df2cc14a819a2b3f9269030f95cbbc042868f2092cd1bdb35988ed6a6329ae26
Sep 10 21:28:16.860271 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:16.860252    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:28:17.028721 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:17.028689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd" event={"ID":"69c7a520-60f8-484f-8dee-2bc726a38d13","Type":"ContainerStarted","Data":"df2cc14a819a2b3f9269030f95cbbc042868f2092cd1bdb35988ed6a6329ae26"}
Sep 10 21:28:18.831707 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:18.831670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:19.490580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.490540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:19.524647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.524614    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"]
Sep 10 21:28:19.525277 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.525226    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4" podUID="db00935c-7a76-4be4-b060-650ad8de0dc9" containerName="payload-pre-processing" containerID="cri-o://4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576" gracePeriod=30
Sep 10 21:28:19.605173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.605125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:19.718010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.717915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:19.786578 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.786551    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:28:19.790359 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.790329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:19.820569 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.820534    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db00935c-7a76-4be4-b060-650ad8de0dc9-plugins-config-volume\") pod \"db00935c-7a76-4be4-b060-650ad8de0dc9\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") "
Sep 10 21:28:19.820739 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.820682    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf8kb\" (UniqueName: \"kubernetes.io/projected/db00935c-7a76-4be4-b060-650ad8de0dc9-kube-api-access-vf8kb\") pod \"db00935c-7a76-4be4-b060-650ad8de0dc9\" (UID: \"db00935c-7a76-4be4-b060-650ad8de0dc9\") "
Sep 10 21:28:19.820885 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.820853    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db00935c-7a76-4be4-b060-650ad8de0dc9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db00935c-7a76-4be4-b060-650ad8de0dc9" (UID: "db00935c-7a76-4be4-b060-650ad8de0dc9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:19.821022 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.821006    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db00935c-7a76-4be4-b060-650ad8de0dc9-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:19.822909 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.822880    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db00935c-7a76-4be4-b060-650ad8de0dc9-kube-api-access-vf8kb" (OuterVolumeSpecName: "kube-api-access-vf8kb") pod "db00935c-7a76-4be4-b060-650ad8de0dc9" (UID: "db00935c-7a76-4be4-b060-650ad8de0dc9"). InnerVolumeSpecName "kube-api-access-vf8kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:19.851781 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.851721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:19.922186 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:19.922139    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf8kb\" (UniqueName: \"kubernetes.io/projected/db00935c-7a76-4be4-b060-650ad8de0dc9-kube-api-access-vf8kb\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:20.045912 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.045818    2583 generic.go:356] "Generic (PLEG): container finished" podID="db00935c-7a76-4be4-b060-650ad8de0dc9" containerID="4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576" exitCode=0
Sep 10 21:28:20.045912 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.045864    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"
Sep 10 21:28:20.045912 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.045894    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4" event={"ID":"db00935c-7a76-4be4-b060-650ad8de0dc9","Type":"ContainerDied","Data":"4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576"}
Sep 10 21:28:20.046181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.045942    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4" event={"ID":"db00935c-7a76-4be4-b060-650ad8de0dc9","Type":"ContainerDied","Data":"5245ba2130608f930642c6fc143f36e3293c9fab3b598a1f50855c5b1b756d63"}
Sep 10 21:28:20.046181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.045965    2583 scope.go:117] "RemoveContainer" containerID="4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576"
Sep 10 21:28:20.047733 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.047711    2583 generic.go:356] "Generic (PLEG): container finished" podID="69c7a520-60f8-484f-8dee-2bc726a38d13" containerID="acd0d68d9d96ed6de798b4eb18e104b81ca9bbd73666b1a787cbf22d3d3ac8ea" exitCode=0
Sep 10 21:28:20.047863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.047777    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd" event={"ID":"69c7a520-60f8-484f-8dee-2bc726a38d13","Type":"ContainerDied","Data":"acd0d68d9d96ed6de798b4eb18e104b81ca9bbd73666b1a787cbf22d3d3ac8ea"}
Sep 10 21:28:20.054449 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.054430    2583 scope.go:117] "RemoveContainer" containerID="4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576"
Sep 10 21:28:20.054756 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:20.054728    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576\": container with ID starting with 4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576 not found: ID does not exist" containerID="4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576"
Sep 10 21:28:20.054890 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.054759    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576"} err="failed to get container status \"4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576\": rpc error: code = NotFound desc = could not find container \"4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576\": container with ID starting with 4b609b7a44f3ed411a094df58b8b7d4dee90611aad8c28946275375608436576 not found: ID does not exist"
Sep 10 21:28:20.074785 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.074753    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"]
Sep 10 21:28:20.078443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.078408    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a6441b-79c64595bb-c6sh4"]
Sep 10 21:28:20.147016 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:20.146979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:21.187438 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:21.187413    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"
Sep 10 21:28:21.233796 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:21.233765    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c7a520-60f8-484f-8dee-2bc726a38d13-maas-api-service-ca\") pod \"69c7a520-60f8-484f-8dee-2bc726a38d13\" (UID: \"69c7a520-60f8-484f-8dee-2bc726a38d13\") "
Sep 10 21:28:21.234155 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:21.234133    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69c7a520-60f8-484f-8dee-2bc726a38d13-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69c7a520-60f8-484f-8dee-2bc726a38d13" (UID: "69c7a520-60f8-484f-8dee-2bc726a38d13"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:21.335054 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:21.334974    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c7a520-60f8-484f-8dee-2bc726a38d13-maas-api-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:21.960730 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:21.960698    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db00935c-7a76-4be4-b060-650ad8de0dc9" path="/var/lib/kubelet/pods/db00935c-7a76-4be4-b060-650ad8de0dc9/volumes"
Sep 10 21:28:22.059912 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:22.059874    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd" event={"ID":"69c7a520-60f8-484f-8dee-2bc726a38d13","Type":"ContainerDied","Data":"df2cc14a819a2b3f9269030f95cbbc042868f2092cd1bdb35988ed6a6329ae26"}
Sep 10 21:28:22.060062 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:22.059931    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df2cc14a819a2b3f9269030f95cbbc042868f2092cd1bdb35988ed6a6329ae26"
Sep 10 21:28:22.060062 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:22.059886    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"
Sep 10 21:28:26.153949 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:26.153912    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:26.252018 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:26.251981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:26.462341 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:26.462248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:26.560223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:26.560186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.024020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.023990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.219849 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.219814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.272625 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.272583    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"]
Sep 10 21:28:27.272880 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.272850    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7" podUID="08a4c5ac-a18f-457a-bcf1-fef288250fd4" containerName="payload-pre-processing" containerID="cri-o://fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba" gracePeriod=30
Sep 10 21:28:27.288221 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288089    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"]
Sep 10 21:28:27.288717 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288693    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db00935c-7a76-4be4-b060-650ad8de0dc9" containerName="payload-pre-processing"
Sep 10 21:28:27.288717 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288718    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="db00935c-7a76-4be4-b060-650ad8de0dc9" containerName="payload-pre-processing"
Sep 10 21:28:27.288901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288757    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69c7a520-60f8-484f-8dee-2bc726a38d13" containerName="revoke-keys"
Sep 10 21:28:27.288901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288766    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="69c7a520-60f8-484f-8dee-2bc726a38d13" containerName="revoke-keys"
Sep 10 21:28:27.288901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288853    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c074bb-314e-43e4-80d9-0fbdb8eef015" containerName="istio-proxy"
Sep 10 21:28:27.288901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288866    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="db00935c-7a76-4be4-b060-650ad8de0dc9" containerName="payload-pre-processing"
Sep 10 21:28:27.288901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.288878    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="69c7a520-60f8-484f-8dee-2bc726a38d13" containerName="revoke-keys"
Sep 10 21:28:27.293001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.292290    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.294311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.294287    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-36b938-openshift-default-dockercfg-k52j7\""
Sep 10 21:28:27.306681 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.306653    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"]
Sep 10 21:28:27.322359 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.322325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.393061 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-token\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393153 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393076    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-envoy\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393153 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393103    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb99666-8595-4571-adb3-c0b63989a8e8-istiod-ca-cert\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393269 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393254    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-podinfo\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393323 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-credential-socket\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393323 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-data\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393430 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-socket\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393430 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pwbj\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-kube-api-access-9pwbj\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.393550 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.393440    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-certs\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.420991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.420790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.481981 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.481931    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"]
Sep 10 21:28:27.488295 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.488262    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-76320447-2xhwd"]
Sep 10 21:28:27.492242 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.492197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.494351 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-certs\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494367    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-token\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494416    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-envoy\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb99666-8595-4571-adb3-c0b63989a8e8-istiod-ca-cert\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494669 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-podinfo\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494669 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494589    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-credential-socket\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494669 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494625    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-data\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494669 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-socket\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494884 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pwbj\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-kube-api-access-9pwbj\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.494884 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.494784    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-certs\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.495052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.495008    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-data\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.495052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.495040    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-socket\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.495229 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.495188    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb99666-8595-4571-adb3-c0b63989a8e8-istiod-ca-cert\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.495286 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.495259    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-credential-socket\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.496813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.496788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-envoy\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.497177 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.497160    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-podinfo\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.501412 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.501389    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-token\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.501622 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.501600    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pwbj\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-kube-api-access-9pwbj\") pod \"e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") " pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.539659 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.539597    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:28:27.558766 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.558736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.595753 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.595716    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rdbc\" (UniqueName: \"kubernetes.io/projected/08a4c5ac-a18f-457a-bcf1-fef288250fd4-kube-api-access-5rdbc\") pod \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") "
Sep 10 21:28:27.595939 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.595782    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08a4c5ac-a18f-457a-bcf1-fef288250fd4-plugins-config-volume\") pod \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\" (UID: \"08a4c5ac-a18f-457a-bcf1-fef288250fd4\") "
Sep 10 21:28:27.596157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.596131    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08a4c5ac-a18f-457a-bcf1-fef288250fd4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "08a4c5ac-a18f-457a-bcf1-fef288250fd4" (UID: "08a4c5ac-a18f-457a-bcf1-fef288250fd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:27.597961 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.597940    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08a4c5ac-a18f-457a-bcf1-fef288250fd4-kube-api-access-5rdbc" (OuterVolumeSpecName: "kube-api-access-5rdbc") pod "08a4c5ac-a18f-457a-bcf1-fef288250fd4" (UID: "08a4c5ac-a18f-457a-bcf1-fef288250fd4"). InnerVolumeSpecName "kube-api-access-5rdbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:27.645468 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.645428    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:27.697563 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.697526    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rdbc\" (UniqueName: \"kubernetes.io/projected/08a4c5ac-a18f-457a-bcf1-fef288250fd4-kube-api-access-5rdbc\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:27.697728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.697568    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08a4c5ac-a18f-457a-bcf1-fef288250fd4-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:28:27.828689 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.828578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.926594 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.926568    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"]
Sep 10 21:28:27.929030 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:28:27.929008    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fb99666_8595_4571_adb3_c0b63989a8e8.slice/crio-70aad36aed73f838863f3d0f032d71a4777fa48d415014780f10663e014c8f96 WatchSource:0}: Error finding container 70aad36aed73f838863f3d0f032d71a4777fa48d415014780f10663e014c8f96: Status 404 returned error can't find the container with id 70aad36aed73f838863f3d0f032d71a4777fa48d415014780f10663e014c8f96
Sep 10 21:28:27.931036 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.931009    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:28:27.931088 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.931078    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:28:27.931132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.931106    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:28:27.960106 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:27.960081    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69c7a520-60f8-484f-8dee-2bc726a38d13" path="/var/lib/kubelet/pods/69c7a520-60f8-484f-8dee-2bc726a38d13/volumes"
Sep 10 21:28:28.091368 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.091272    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" event={"ID":"3fb99666-8595-4571-adb3-c0b63989a8e8","Type":"ContainerStarted","Data":"6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b"}
Sep 10 21:28:28.091368 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.091340    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" event={"ID":"3fb99666-8595-4571-adb3-c0b63989a8e8","Type":"ContainerStarted","Data":"70aad36aed73f838863f3d0f032d71a4777fa48d415014780f10663e014c8f96"}
Sep 10 21:28:28.092630 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.092605    2583 generic.go:356] "Generic (PLEG): container finished" podID="08a4c5ac-a18f-457a-bcf1-fef288250fd4" containerID="fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba" exitCode=0
Sep 10 21:28:28.092759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.092644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7" event={"ID":"08a4c5ac-a18f-457a-bcf1-fef288250fd4","Type":"ContainerDied","Data":"fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba"}
Sep 10 21:28:28.092759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.092647    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"
Sep 10 21:28:28.092759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.092674    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7" event={"ID":"08a4c5ac-a18f-457a-bcf1-fef288250fd4","Type":"ContainerDied","Data":"81c2732892197e99bad82942b8e0c7e3b33df31ebcf3c22de14275a35f86d486"}
Sep 10 21:28:28.092759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.092701    2583 scope.go:117] "RemoveContainer" containerID="fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba"
Sep 10 21:28:28.101199 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.101172    2583 scope.go:117] "RemoveContainer" containerID="fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba"
Sep 10 21:28:28.101512 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:28:28.101475    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba\": container with ID starting with fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba not found: ID does not exist" containerID="fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba"
Sep 10 21:28:28.101599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.101541    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba"} err="failed to get container status \"fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba\": rpc error: code = NotFound desc = could not find container \"fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba\": container with ID starting with fdb3a93314b3d2a0b59b77a013a2ea5a5ef2fc77545e0ffb660ca030aabee8ba not found: ID does not exist"
Sep 10 21:28:28.111559 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.111490    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podStartSLOduration=1.111477059 podStartE2EDuration="1.111477059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:28.109272651 +0000 UTC m=+1098.636236997" watchObservedRunningTime="2026-09-10 21:28:28.111477059 +0000 UTC m=+1098.638441405"
Sep 10 21:28:28.124255 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.124225    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"]
Sep 10 21:28:28.126643 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.126618    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-76320447-59d58cdd4b-wmwj7"]
Sep 10 21:28:28.646498 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.646461    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:28.651686 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:28.651660    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:29.099176 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:29.099089    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:29.100119 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:29.100103    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:29.961544 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:29.961492    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08a4c5ac-a18f-457a-bcf1-fef288250fd4" path="/var/lib/kubelet/pods/08a4c5ac-a18f-457a-bcf1-fef288250fd4/volumes"
Sep 10 21:28:29.977390 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:29.977347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:30.077580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:30.077551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:30.978600 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:30.978565    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:31.322810 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:31.322637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:31.425778 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:31.425732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:31.747236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:31.747205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:31.854317 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:31.854284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:31.951916 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:31.951881    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:32.086906 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.086813    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"]
Sep 10 21:28:32.087368 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.087203    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a4c5ac-a18f-457a-bcf1-fef288250fd4" containerName="payload-pre-processing"
Sep 10 21:28:32.087368 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.087218    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a4c5ac-a18f-457a-bcf1-fef288250fd4" containerName="payload-pre-processing"
Sep 10 21:28:32.087368 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.087292    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a4c5ac-a18f-457a-bcf1-fef288250fd4" containerName="payload-pre-processing"
Sep 10 21:28:32.090274 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.090248    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.092790 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.092770    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-36b938-dockercfg-ld5s2\""
Sep 10 21:28:32.097795 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.097709    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-36b938\""
Sep 10 21:28:32.109041 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.108999    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"]
Sep 10 21:28:32.143678 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.143645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4848ca30-a444-4635-9a5c-0680aefb235a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.143843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.143768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72dxp\" (UniqueName: \"kubernetes.io/projected/4848ca30-a444-4635-9a5c-0680aefb235a-kube-api-access-72dxp\") pod \"payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.244405 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.244370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72dxp\" (UniqueName: \"kubernetes.io/projected/4848ca30-a444-4635-9a5c-0680aefb235a-kube-api-access-72dxp\") pod \"payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.244627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.244439    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4848ca30-a444-4635-9a5c-0680aefb235a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.245335 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.245310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4848ca30-a444-4635-9a5c-0680aefb235a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.253790 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.253756    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72dxp\" (UniqueName: \"kubernetes.io/projected/4848ca30-a444-4635-9a5c-0680aefb235a-kube-api-access-72dxp\") pod \"payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.403164 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.403131    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:32.684786 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:32.684763    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"]
Sep 10 21:28:32.687272 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:28:32.687243    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4848ca30_a444_4635_9a5c_0680aefb235a.slice/crio-a8a15c948b026c370bf12326078b3b10e60ab80df5a4db984466191ae3add56f WatchSource:0}: Error finding container a8a15c948b026c370bf12326078b3b10e60ab80df5a4db984466191ae3add56f: Status 404 returned error can't find the container with id a8a15c948b026c370bf12326078b3b10e60ab80df5a4db984466191ae3add56f
Sep 10 21:28:33.121466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:33.121372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb" event={"ID":"4848ca30-a444-4635-9a5c-0680aefb235a","Type":"ContainerStarted","Data":"250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6"}
Sep 10 21:28:33.121466 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:33.121438    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb" event={"ID":"4848ca30-a444-4635-9a5c-0680aefb235a","Type":"ContainerStarted","Data":"a8a15c948b026c370bf12326078b3b10e60ab80df5a4db984466191ae3add56f"}
Sep 10 21:28:33.121886 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:33.121531    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:33.162167 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:33.162111    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb" podStartSLOduration=1.162092519 podStartE2EDuration="1.162092519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:33.160106871 +0000 UTC m=+1103.687071217" watchObservedRunningTime="2026-09-10 21:28:33.162092519 +0000 UTC m=+1103.689056865"
Sep 10 21:28:33.967048 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:33.967017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:34.037905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:34.037873    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:34.225223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:34.225132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:34.565098 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:34.565007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:34.666191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:34.666155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:34.766540 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:34.766490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:41.027738 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:41.027699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:41.140981 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:41.140945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:41.228398 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:41.228306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:41.363994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:41.363905    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:44.129043 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:44.129008    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:28:45.141664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:45.141630    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:28:45.142121 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:45.141684    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:28:46.188516 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:46.188466    2583 generic.go:356] "Generic (PLEG): container finished" podID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerID="6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b" exitCode=0
Sep 10 21:28:46.188943 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:46.188537    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" event={"ID":"3fb99666-8595-4571-adb3-c0b63989a8e8","Type":"ContainerDied","Data":"6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b"}
Sep 10 21:28:46.188989 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:46.188940    2583 scope.go:117] "RemoveContainer" containerID="6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b"
Sep 10 21:28:46.190014 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:46.189987    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:28:46.190080 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:46.190056    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:28:46.190117 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:46.190088    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:28:47.186631 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:47.186585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:47.199116 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:47.199085    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" event={"ID":"3fb99666-8595-4571-adb3-c0b63989a8e8","Type":"ContainerStarted","Data":"b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382"}
Sep 10 21:28:47.285538 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:47.285477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:47.646595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:47.646548    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:47.646816 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:47.646798    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:28:47.646869 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:47.646842    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 10 21:28:48.092636 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:48.092518    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:48.431162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:48.431083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:48.646009 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:48.645967    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:28:48.646190 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:48.646039    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 10 21:28:48.742332 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:48.742238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:49.646065 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:49.646028    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:28:49.646478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:49.646084    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 10 21:28:50.646191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:50.646156    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:28:50.646604 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:50.646224    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 10 21:28:51.479880 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:51.479842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:51.573380 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:51.573344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:51.646369 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:51.646330    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 10 21:28:51.646757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:51.646405    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 10 21:28:52.651573 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:52.651540    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:52.652351 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:52.652321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:52.652970 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:52.652953    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:28:52.700468 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:52.700433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:52.769785 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:52.769749    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.363722 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.363688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.468581 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.468546    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.495433 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.492297    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"]
Sep 10 21:28:53.502063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.502022    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.510399 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.510378    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-bd7b8a\""
Sep 10 21:28:53.510957 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.510933    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-bd7b8a-dockercfg-5c799\""
Sep 10 21:28:53.513584 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.513559    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"]
Sep 10 21:28:53.565254 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.565222    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"]
Sep 10 21:28:53.567563 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.567541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881f31d8-d421-48d0-9978-9ea26f413b44-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.567652 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.567635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7wpd\" (UniqueName: \"kubernetes.io/projected/881f31d8-d421-48d0-9978-9ea26f413b44-kube-api-access-m7wpd\") pod \"payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.569036 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.569020    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:53.579746 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.579723    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"]
Sep 10 21:28:53.668570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.668495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a92b351-087e-4c66-8660-c0df76ec3ba7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:53.669010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.668620    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7wpd\" (UniqueName: \"kubernetes.io/projected/881f31d8-d421-48d0-9978-9ea26f413b44-kube-api-access-m7wpd\") pod \"payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.669010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.668673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk6bs\" (UniqueName: \"kubernetes.io/projected/8a92b351-087e-4c66-8660-c0df76ec3ba7-kube-api-access-rk6bs\") pod \"payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:53.669010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.668762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881f31d8-d421-48d0-9978-9ea26f413b44-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.669488 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.669470    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881f31d8-d421-48d0-9978-9ea26f413b44-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.676404 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.676379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7wpd\" (UniqueName: \"kubernetes.io/projected/881f31d8-d421-48d0-9978-9ea26f413b44-kube-api-access-m7wpd\") pod \"payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.769724 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.769688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a92b351-087e-4c66-8660-c0df76ec3ba7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:53.769933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.769742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk6bs\" (UniqueName: \"kubernetes.io/projected/8a92b351-087e-4c66-8660-c0df76ec3ba7-kube-api-access-rk6bs\") pod \"payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:53.770478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.770457    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a92b351-087e-4c66-8660-c0df76ec3ba7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:53.780001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.779973    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk6bs\" (UniqueName: \"kubernetes.io/projected/8a92b351-087e-4c66-8660-c0df76ec3ba7-kube-api-access-rk6bs\") pod \"payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:53.812194 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.812155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.814057 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.814032    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:53.883372 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:53.883104    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:54.088970 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.088917    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"]
Sep 10 21:28:54.091737 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:28:54.091712    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881f31d8_d421_48d0_9978_9ea26f413b44.slice/crio-597adc54852fb8b7882c1d88673af492e77a6192b3802afa03718faf67f9bad0 WatchSource:0}: Error finding container 597adc54852fb8b7882c1d88673af492e77a6192b3802afa03718faf67f9bad0: Status 404 returned error can't find the container with id 597adc54852fb8b7882c1d88673af492e77a6192b3802afa03718faf67f9bad0
Sep 10 21:28:54.159187 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.159159    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"]
Sep 10 21:28:54.161740 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:28:54.161708    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a92b351_087e_4c66_8660_c0df76ec3ba7.slice/crio-9bf2d8010754d853f6678f618c89bb6db34d0d7c5dfc59d1ac5ca8c0153b6952 WatchSource:0}: Error finding container 9bf2d8010754d853f6678f618c89bb6db34d0d7c5dfc59d1ac5ca8c0153b6952: Status 404 returned error can't find the container with id 9bf2d8010754d853f6678f618c89bb6db34d0d7c5dfc59d1ac5ca8c0153b6952
Sep 10 21:28:54.247224 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.247188    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6" event={"ID":"8a92b351-087e-4c66-8660-c0df76ec3ba7","Type":"ContainerStarted","Data":"9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697"}
Sep 10 21:28:54.247400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.247260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6" event={"ID":"8a92b351-087e-4c66-8660-c0df76ec3ba7","Type":"ContainerStarted","Data":"9bf2d8010754d853f6678f618c89bb6db34d0d7c5dfc59d1ac5ca8c0153b6952"}
Sep 10 21:28:54.247400 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.247319    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:28:54.249020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.248997    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz" event={"ID":"881f31d8-d421-48d0-9978-9ea26f413b44","Type":"ContainerStarted","Data":"fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9"}
Sep 10 21:28:54.249153 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.249039    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:28:54.249153 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.249061    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz" event={"ID":"881f31d8-d421-48d0-9978-9ea26f413b44","Type":"ContainerStarted","Data":"597adc54852fb8b7882c1d88673af492e77a6192b3802afa03718faf67f9bad0"}
Sep 10 21:28:54.265236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.264781    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6" podStartSLOduration=1.264761966 podStartE2EDuration="1.264761966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:54.264094865 +0000 UTC m=+1124.791059217" watchObservedRunningTime="2026-09-10 21:28:54.264761966 +0000 UTC m=+1124.791726313"
Sep 10 21:28:54.283645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.283594    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz" podStartSLOduration=1.283577167 podStartE2EDuration="1.283577167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:54.282408564 +0000 UTC m=+1124.809372911" watchObservedRunningTime="2026-09-10 21:28:54.283577167 +0000 UTC m=+1124.810541514"
Sep 10 21:28:54.610144 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.610034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:54.914319 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:54.914282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:55.277396 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:55.277308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:55.555840 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:55.555758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:55.808118 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:55.808029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:56.025352 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:56.025297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:58.531646 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:58.531612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:58.628122 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:58.628091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:59.750121 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:59.750074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:59.847271 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:28:59.847217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:00.060695 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.060602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:00.375424 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.375311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:00.471863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.471828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:00.504268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.504232    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"]
Sep 10 21:29:00.507885 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.507862    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:00.509841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.509819    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-1f3879\""
Sep 10 21:29:00.510018 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.509990    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-1f3879-dockercfg-hr6bn\""
Sep 10 21:29:00.524994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.524972    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"]
Sep 10 21:29:00.546404 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.546371    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7skk\" (UniqueName: \"kubernetes.io/projected/73bb8431-e77c-41c1-8f63-ccfab23bc730-kube-api-access-l7skk\") pod \"payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:00.546587 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.546483    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bb8431-e77c-41c1-8f63-ccfab23bc730-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:00.647608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.647571    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7skk\" (UniqueName: \"kubernetes.io/projected/73bb8431-e77c-41c1-8f63-ccfab23bc730-kube-api-access-l7skk\") pod \"payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:00.647787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.647629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bb8431-e77c-41c1-8f63-ccfab23bc730-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:00.648422 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.648402    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bb8431-e77c-41c1-8f63-ccfab23bc730-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:00.658135 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.658109    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7skk\" (UniqueName: \"kubernetes.io/projected/73bb8431-e77c-41c1-8f63-ccfab23bc730-kube-api-access-l7skk\") pod \"payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:00.819049 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:00.819015    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:01.091166 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:01.091072    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"]
Sep 10 21:29:01.095872 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:29:01.095843    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bb8431_e77c_41c1_8f63_ccfab23bc730.slice/crio-b592b149fe70238a948da407504ffd9ed7953dfb5593135760c08d8b990b001c WatchSource:0}: Error finding container b592b149fe70238a948da407504ffd9ed7953dfb5593135760c08d8b990b001c: Status 404 returned error can't find the container with id b592b149fe70238a948da407504ffd9ed7953dfb5593135760c08d8b990b001c
Sep 10 21:29:01.289646 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:01.289608    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx" event={"ID":"73bb8431-e77c-41c1-8f63-ccfab23bc730","Type":"ContainerStarted","Data":"feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4"}
Sep 10 21:29:01.289824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:01.289663    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx" event={"ID":"73bb8431-e77c-41c1-8f63-ccfab23bc730","Type":"ContainerStarted","Data":"b592b149fe70238a948da407504ffd9ed7953dfb5593135760c08d8b990b001c"}
Sep 10 21:29:01.289824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:01.289758    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:01.305928 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:01.305877    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx" podStartSLOduration=1.305862613 podStartE2EDuration="1.305862613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:01.304039221 +0000 UTC m=+1131.831003571" watchObservedRunningTime="2026-09-10 21:29:01.305862613 +0000 UTC m=+1131.832826960"
Sep 10 21:29:05.256929 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:05.256892    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:29:05.257346 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:05.256973    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:29:12.296988 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:12.296958    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:29:13.802137 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:13.802098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:16.402082 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:16.402044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:16.730976 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:16.730879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:17.037592 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:17.037477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:17.979863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:17.979830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:19.801776 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:19.801721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:19.902036 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:19.902004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:20.918230 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:20.912146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:21.820738 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:21.820707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:21.927523 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:21.927464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:22.354103 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.354067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:22.457485 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.457450    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:22.554490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.554455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:22.578699 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.578664    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"]
Sep 10 21:29:22.583440 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.583420    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:22.585140 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.585116    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-5e6835\""
Sep 10 21:29:22.585260 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.585174    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-5e6835-dockercfg-s46mb\""
Sep 10 21:29:22.591405 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.591383    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"]
Sep 10 21:29:22.764758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.764726    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/755f26c0-c5dd-435f-9ee8-e945f3caebf1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:22.764758 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.764767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqw8t\" (UniqueName: \"kubernetes.io/projected/755f26c0-c5dd-435f-9ee8-e945f3caebf1-kube-api-access-pqw8t\") pod \"payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:22.865754 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.865715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqw8t\" (UniqueName: \"kubernetes.io/projected/755f26c0-c5dd-435f-9ee8-e945f3caebf1-kube-api-access-pqw8t\") pod \"payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:22.865963 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.865843    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/755f26c0-c5dd-435f-9ee8-e945f3caebf1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:22.866572 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.866490    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/755f26c0-c5dd-435f-9ee8-e945f3caebf1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:22.878626 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.878605    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqw8t\" (UniqueName: \"kubernetes.io/projected/755f26c0-c5dd-435f-9ee8-e945f3caebf1-kube-api-access-pqw8t\") pod \"payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:22.893384 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.893360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:22.895284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:22.895268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:23.167944 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:23.167890    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"]
Sep 10 21:29:23.170881 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:29:23.170852    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755f26c0_c5dd_435f_9ee8_e945f3caebf1.slice/crio-31caabad7ad3c9866ae585658e3aa63dda1a8048fb21c1b3647858aee0cf64e8 WatchSource:0}: Error finding container 31caabad7ad3c9866ae585658e3aa63dda1a8048fb21c1b3647858aee0cf64e8: Status 404 returned error can't find the container with id 31caabad7ad3c9866ae585658e3aa63dda1a8048fb21c1b3647858aee0cf64e8
Sep 10 21:29:23.406125 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:23.406087    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf" event={"ID":"755f26c0-c5dd-435f-9ee8-e945f3caebf1","Type":"ContainerStarted","Data":"3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee"}
Sep 10 21:29:23.406307 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:23.406140    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf" event={"ID":"755f26c0-c5dd-435f-9ee8-e945f3caebf1","Type":"ContainerStarted","Data":"31caabad7ad3c9866ae585658e3aa63dda1a8048fb21c1b3647858aee0cf64e8"}
Sep 10 21:29:23.406307 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:23.406241    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:23.426453 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:23.426342    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf" podStartSLOduration=1.426322677 podStartE2EDuration="1.426322677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:23.424052498 +0000 UTC m=+1153.951016845" watchObservedRunningTime="2026-09-10 21:29:23.426322677 +0000 UTC m=+1153.953287024"
Sep 10 21:29:23.446617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:23.446580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:23.748875 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:23.748768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:34.414304 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:34.414219    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:29:35.973284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:35.973253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:44.668495 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:44.668464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:54.169033 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:54.168976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:54.485343 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:54.485251    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:54.866680 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:54.866583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:57.916063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:57.916031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:58.015409 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:58.015377    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:59.120714 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.120683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:59.224289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.224251    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:59.422732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.422697    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:59.622954 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.622920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:59.727052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.726965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:59.733864 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.733839    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"]
Sep 10 21:29:59.737168 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.737151    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:29:59.739120 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.739098    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7ce89dd6\""
Sep 10 21:29:59.749376 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.749352    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"]
Sep 10 21:29:59.764644 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.764613    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"]
Sep 10 21:29:59.768712 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.768683    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:29:59.770927 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.770905    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7ce89dd6-dockercfg-lcc8g\""
Sep 10 21:29:59.778133 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.778108    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"]
Sep 10 21:29:59.922372 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.922330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcsmx\" (UniqueName: \"kubernetes.io/projected/8facb9f1-7c89-4986-a398-cc416c118612-kube-api-access-lcsmx\") pod \"payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:29:59.922616 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.922407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce05c1e9-ad62-4672-83a4-2234da27c068-plugins-config-volume\") pod \"payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") " pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:29:59.922616 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.922457    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6lnp\" (UniqueName: \"kubernetes.io/projected/ce05c1e9-ad62-4672-83a4-2234da27c068-kube-api-access-t6lnp\") pod \"payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") " pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:29:59.922616 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:29:59.922560    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8facb9f1-7c89-4986-a398-cc416c118612-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:00.023952 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.023853    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6lnp\" (UniqueName: \"kubernetes.io/projected/ce05c1e9-ad62-4672-83a4-2234da27c068-kube-api-access-t6lnp\") pod \"payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") " pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:30:00.023952 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.023934    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8facb9f1-7c89-4986-a398-cc416c118612-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:00.024200 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.024029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcsmx\" (UniqueName: \"kubernetes.io/projected/8facb9f1-7c89-4986-a398-cc416c118612-kube-api-access-lcsmx\") pod \"payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:00.024200 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.024085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce05c1e9-ad62-4672-83a4-2234da27c068-plugins-config-volume\") pod \"payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") " pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:30:00.024793 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.024771    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8facb9f1-7c89-4986-a398-cc416c118612-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:00.025045 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.025024    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce05c1e9-ad62-4672-83a4-2234da27c068-plugins-config-volume\") pod \"payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") " pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:30:00.032548 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.032496    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcsmx\" (UniqueName: \"kubernetes.io/projected/8facb9f1-7c89-4986-a398-cc416c118612-kube-api-access-lcsmx\") pod \"payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:00.032668 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.032645    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6lnp\" (UniqueName: \"kubernetes.io/projected/ce05c1e9-ad62-4672-83a4-2234da27c068-kube-api-access-t6lnp\") pod \"payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") " pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:30:00.047295 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.047259    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:00.080538 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.080481    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:30:00.144150 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.144069    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"]
Sep 10 21:30:00.156294 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.155876    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"]
Sep 10 21:30:00.157269 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.156582    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"
Sep 10 21:30:00.159724 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.158648    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:30:00.162997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.161018    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"]
Sep 10 21:30:00.162997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.161736    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"
Sep 10 21:30:00.165052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.164933    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"]
Sep 10 21:30:00.165052 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.165035    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"
Sep 10 21:30:00.166493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.166470    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"]
Sep 10 21:30:00.179047 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.179018    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"]
Sep 10 21:30:00.179266 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.179078    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"
Sep 10 21:30:00.190449 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.190410    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"
Sep 10 21:30:00.199788 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.199766    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"
Sep 10 21:30:00.329662 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.329633    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"]
Sep 10 21:30:00.332753 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:30:00.332721    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8facb9f1_7c89_4986_a398_cc416c118612.slice/crio-ac1e004cd5f259a28319be1e8cd524449168845bdc056d0090a115366b2f7b98 WatchSource:0}: Error finding container ac1e004cd5f259a28319be1e8cd524449168845bdc056d0090a115366b2f7b98: Status 404 returned error can't find the container with id ac1e004cd5f259a28319be1e8cd524449168845bdc056d0090a115366b2f7b98
Sep 10 21:30:00.369301 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.369275    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"]
Sep 10 21:30:00.371170 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:30:00.371144    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce05c1e9_ad62_4672_83a4_2234da27c068.slice/crio-01d5f12a8734282d5916d3677a78e506b027c72e3d51b7957591754a212e663c WatchSource:0}: Error finding container 01d5f12a8734282d5916d3677a78e506b027c72e3d51b7957591754a212e663c: Status 404 returned error can't find the container with id 01d5f12a8734282d5916d3677a78e506b027c72e3d51b7957591754a212e663c
Sep 10 21:30:00.464939 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.464907    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"]
Sep 10 21:30:00.467331 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:30:00.467299    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb664ed25_3ddf_4f5f_a253_c5191b47e870.slice/crio-2099273fd25eb7801a7728ed2532b837f75a34f96482087cd9e2aa3c678181ae WatchSource:0}: Error finding container 2099273fd25eb7801a7728ed2532b837f75a34f96482087cd9e2aa3c678181ae: Status 404 returned error can't find the container with id 2099273fd25eb7801a7728ed2532b837f75a34f96482087cd9e2aa3c678181ae
Sep 10 21:30:00.488644 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.488573    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"]
Sep 10 21:30:00.491891 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:30:00.491854    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381bf1d3_7259_48c7_8b9f_dc73802a5c38.slice/crio-b9391397ce777ab729bc34cc31ee4e9bbd43df77d726b36ca76dfd2f82b38568 WatchSource:0}: Error finding container b9391397ce777ab729bc34cc31ee4e9bbd43df77d726b36ca76dfd2f82b38568: Status 404 returned error can't find the container with id b9391397ce777ab729bc34cc31ee4e9bbd43df77d726b36ca76dfd2f82b38568
Sep 10 21:30:00.492197 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.492080    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"]
Sep 10 21:30:00.501730 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:30:00.501705    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac1895a_72df_4f98_8709_2e923eaebacb.slice/crio-b3adaac5390b58e47b4944c9d3c541f9d27945646a6618d5745167378b3e4dcc WatchSource:0}: Error finding container b3adaac5390b58e47b4944c9d3c541f9d27945646a6618d5745167378b3e4dcc: Status 404 returned error can't find the container with id b3adaac5390b58e47b4944c9d3c541f9d27945646a6618d5745167378b3e4dcc
Sep 10 21:30:00.596624 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.596580    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9" event={"ID":"8facb9f1-7c89-4986-a398-cc416c118612","Type":"ContainerStarted","Data":"00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208"}
Sep 10 21:30:00.596836 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.596650    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9" event={"ID":"8facb9f1-7c89-4986-a398-cc416c118612","Type":"ContainerStarted","Data":"ac1e004cd5f259a28319be1e8cd524449168845bdc056d0090a115366b2f7b98"}
Sep 10 21:30:00.596836 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.596673    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:00.598034 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.598009    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk" event={"ID":"b664ed25-3ddf-4f5f-a253-c5191b47e870","Type":"ContainerStarted","Data":"2099273fd25eb7801a7728ed2532b837f75a34f96482087cd9e2aa3c678181ae"}
Sep 10 21:30:00.599636 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.599609    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4" event={"ID":"ce05c1e9-ad62-4672-83a4-2234da27c068","Type":"ContainerStarted","Data":"570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7"}
Sep 10 21:30:00.599793 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.599664    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4" event={"ID":"ce05c1e9-ad62-4672-83a4-2234da27c068","Type":"ContainerStarted","Data":"01d5f12a8734282d5916d3677a78e506b027c72e3d51b7957591754a212e663c"}
Sep 10 21:30:00.599793 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.599696    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:30:00.601000 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.600976    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" event={"ID":"381bf1d3-7259-48c7-8b9f-dc73802a5c38","Type":"ContainerStarted","Data":"b9391397ce777ab729bc34cc31ee4e9bbd43df77d726b36ca76dfd2f82b38568"}
Sep 10 21:30:00.602225 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.602207    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h" event={"ID":"dac1895a-72df-4f98-8709-2e923eaebacb","Type":"ContainerStarted","Data":"b3adaac5390b58e47b4944c9d3c541f9d27945646a6618d5745167378b3e4dcc"}
Sep 10 21:30:00.613246 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.613164    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9" podStartSLOduration=1.6131473870000002 podStartE2EDuration="1.613147387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:00.612025667 +0000 UTC m=+1191.138990020" watchObservedRunningTime="2026-09-10 21:30:00.613147387 +0000 UTC m=+1191.140111734"
Sep 10 21:30:00.629699 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:00.629635    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4" podStartSLOduration=1.6296162440000002 podStartE2EDuration="1.629616244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:00.628495702 +0000 UTC m=+1191.155460050" watchObservedRunningTime="2026-09-10 21:30:00.629616244 +0000 UTC m=+1191.156580627"
Sep 10 21:30:01.610441 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:01.610400    2583 generic.go:356] "Generic (PLEG): container finished" podID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerID="dc40d73ce7e1f41b51ea299051606530b56a3c6ba1b2f995e2abae9934342b35" exitCode=7
Sep 10 21:30:01.610901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:01.610532    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" event={"ID":"381bf1d3-7259-48c7-8b9f-dc73802a5c38","Type":"ContainerDied","Data":"dc40d73ce7e1f41b51ea299051606530b56a3c6ba1b2f995e2abae9934342b35"}
Sep 10 21:30:01.610901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:01.610835    2583 scope.go:117] "RemoveContainer" containerID="dc40d73ce7e1f41b51ea299051606530b56a3c6ba1b2f995e2abae9934342b35"
Sep 10 21:30:01.614127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:01.614031    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h" event={"ID":"dac1895a-72df-4f98-8709-2e923eaebacb","Type":"ContainerStarted","Data":"a0d8677377166d789b9c2130312de907d8896dc7f2e37f746b1e758eaa179b6c"}
Sep 10 21:30:01.617315 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:01.616826    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk" event={"ID":"b664ed25-3ddf-4f5f-a253-c5191b47e870","Type":"ContainerStarted","Data":"4f9947f166fdd506a28d3fde5ccdf3c61e54f0df718ef533dfa6907133483f08"}
Sep 10 21:30:01.655053 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:01.654980    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h" podStartSLOduration=1.65495988 podStartE2EDuration="1.65495988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:01.654306924 +0000 UTC m=+1192.181271272" watchObservedRunningTime="2026-09-10 21:30:01.65495988 +0000 UTC m=+1192.181924227"
Sep 10 21:30:02.623919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.623822    2583 generic.go:356] "Generic (PLEG): container finished" podID="b664ed25-3ddf-4f5f-a253-c5191b47e870" containerID="4f9947f166fdd506a28d3fde5ccdf3c61e54f0df718ef533dfa6907133483f08" exitCode=0
Sep 10 21:30:02.623919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.623906    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk" event={"ID":"b664ed25-3ddf-4f5f-a253-c5191b47e870","Type":"ContainerDied","Data":"4f9947f166fdd506a28d3fde5ccdf3c61e54f0df718ef533dfa6907133483f08"}
Sep 10 21:30:02.626132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.626091    2583 generic.go:356] "Generic (PLEG): container finished" podID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerID="ca8ee6232dd1dfc42a962e994e7e832e9ac41857f2381d2ab715b1ecf2881633" exitCode=7
Sep 10 21:30:02.626233 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.626167    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" event={"ID":"381bf1d3-7259-48c7-8b9f-dc73802a5c38","Type":"ContainerDied","Data":"ca8ee6232dd1dfc42a962e994e7e832e9ac41857f2381d2ab715b1ecf2881633"}
Sep 10 21:30:02.626289 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.626231    2583 scope.go:117] "RemoveContainer" containerID="dc40d73ce7e1f41b51ea299051606530b56a3c6ba1b2f995e2abae9934342b35"
Sep 10 21:30:02.626454 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.626431    2583 scope.go:117] "RemoveContainer" containerID="ca8ee6232dd1dfc42a962e994e7e832e9ac41857f2381d2ab715b1ecf2881633"
Sep 10 21:30:02.626796 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:30:02.626766    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg_odh-ai-gateway-infra(381bf1d3-7259-48c7-8b9f-dc73802a5c38)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38"
Sep 10 21:30:02.628648 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.628627    2583 generic.go:356] "Generic (PLEG): container finished" podID="dac1895a-72df-4f98-8709-2e923eaebacb" containerID="a0d8677377166d789b9c2130312de907d8896dc7f2e37f746b1e758eaa179b6c" exitCode=0
Sep 10 21:30:02.628728 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.628706    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h" event={"ID":"dac1895a-72df-4f98-8709-2e923eaebacb","Type":"ContainerDied","Data":"a0d8677377166d789b9c2130312de907d8896dc7f2e37f746b1e758eaa179b6c"}
Sep 10 21:30:02.775479 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:02.775451    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"
Sep 10 21:30:03.636780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:03.636746    2583 scope.go:117] "RemoveContainer" containerID="ca8ee6232dd1dfc42a962e994e7e832e9ac41857f2381d2ab715b1ecf2881633"
Sep 10 21:30:03.637243 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:30:03.637021    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg_odh-ai-gateway-infra(381bf1d3-7259-48c7-8b9f-dc73802a5c38)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38"
Sep 10 21:30:03.637919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:03.637899    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"
Sep 10 21:30:03.638049 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:03.637917    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk" event={"ID":"b664ed25-3ddf-4f5f-a253-c5191b47e870","Type":"ContainerDied","Data":"2099273fd25eb7801a7728ed2532b837f75a34f96482087cd9e2aa3c678181ae"}
Sep 10 21:30:03.638049 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:03.637971    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2099273fd25eb7801a7728ed2532b837f75a34f96482087cd9e2aa3c678181ae"
Sep 10 21:30:03.760710 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:03.760683    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"
Sep 10 21:30:04.645123 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:04.645092    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"
Sep 10 21:30:04.645563 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:04.645091    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h" event={"ID":"dac1895a-72df-4f98-8709-2e923eaebacb","Type":"ContainerDied","Data":"b3adaac5390b58e47b4944c9d3c541f9d27945646a6618d5745167378b3e4dcc"}
Sep 10 21:30:04.645563 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:04.645240    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3adaac5390b58e47b4944c9d3c541f9d27945646a6618d5745167378b3e4dcc"
Sep 10 21:30:09.988807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:09.988780    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:30:09.990208 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:09.990188    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:30:11.619623 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:11.619591    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:30:11.620087 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:11.619670    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:30:12.554603 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:12.554562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:30:15.954627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:15.954592    2583 scope.go:117] "RemoveContainer" containerID="ca8ee6232dd1dfc42a962e994e7e832e9ac41857f2381d2ab715b1ecf2881633"
Sep 10 21:30:16.715410 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:16.715373    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" event={"ID":"381bf1d3-7259-48c7-8b9f-dc73802a5c38","Type":"ContainerStarted","Data":"0e6f9ff43b23145719c356e767c33d7318094506fb30a32f8473728499ac7539"}
Sep 10 21:30:16.732637 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:16.732580    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" podStartSLOduration=16.732562704 podStartE2EDuration="16.732562704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:30:16.731623531 +0000 UTC m=+1207.258587879" watchObservedRunningTime="2026-09-10 21:30:16.732562704 +0000 UTC m=+1207.259527088"
Sep 10 21:30:17.722965 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:17.722934    2583 generic.go:356] "Generic (PLEG): container finished" podID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerID="0e6f9ff43b23145719c356e767c33d7318094506fb30a32f8473728499ac7539" exitCode=0
Sep 10 21:30:17.723419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:17.723010    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" event={"ID":"381bf1d3-7259-48c7-8b9f-dc73802a5c38","Type":"ContainerDied","Data":"0e6f9ff43b23145719c356e767c33d7318094506fb30a32f8473728499ac7539"}
Sep 10 21:30:17.723419 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:17.723088    2583 scope.go:117] "RemoveContainer" containerID="ca8ee6232dd1dfc42a962e994e7e832e9ac41857f2381d2ab715b1ecf2881633"
Sep 10 21:30:17.748484 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:17.748455    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"]
Sep 10 21:30:18.864673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:18.864641    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"
Sep 10 21:30:19.736043 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:19.736011    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"
Sep 10 21:30:19.736228 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:19.736011    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg" event={"ID":"381bf1d3-7259-48c7-8b9f-dc73802a5c38","Type":"ContainerDied","Data":"b9391397ce777ab729bc34cc31ee4e9bbd43df77d726b36ca76dfd2f82b38568"}
Sep 10 21:30:19.736228 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:19.736188    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9391397ce777ab729bc34cc31ee4e9bbd43df77d726b36ca76dfd2f82b38568"
Sep 10 21:30:19.763872 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:19.763844    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"]
Sep 10 21:30:19.771446 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:19.771421    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-7ce89dd6-29817930-f4jjg"]
Sep 10 21:30:19.961295 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:19.961262    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" path="/var/lib/kubelet/pods/381bf1d3-7259-48c7-8b9f-dc73802a5c38/volumes"
Sep 10 21:30:22.853025 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:22.852986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:30:50.341496 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:30:50.341463    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:07.706641 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:07.706604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:17.524014 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.523976    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"]
Sep 10 21:31:17.524472 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524457    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b664ed25-3ddf-4f5f-a253-c5191b47e870" containerName="cleanup"
Sep 10 21:31:17.524532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524475    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b664ed25-3ddf-4f5f-a253-c5191b47e870" containerName="cleanup"
Sep 10 21:31:17.524532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524488    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524493    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524516    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac1895a-72df-4f98-8709-2e923eaebacb" containerName="cleanup"
Sep 10 21:31:17.524532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524522    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac1895a-72df-4f98-8709-2e923eaebacb" containerName="cleanup"
Sep 10 21:31:17.524532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524527    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524532 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524533    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524543    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524548    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524612    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524621    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b664ed25-3ddf-4f5f-a253-c5191b47e870" containerName="cleanup"
Sep 10 21:31:17.524755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524630    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:31:17.524755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.524637    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dac1895a-72df-4f98-8709-2e923eaebacb" containerName="cleanup"
Sep 10 21:31:17.527748 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.527732    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"
Sep 10 21:31:17.529716 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.529695    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:31:17.535805 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.535597    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"]
Sep 10 21:31:17.575734 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.575701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b056e2ac-15c4-4c5e-850e-7140f1985c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx\" (UID: \"b056e2ac-15c4-4c5e-850e-7140f1985c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"
Sep 10 21:31:17.676026 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.675990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:17.676297 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.676274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b056e2ac-15c4-4c5e-850e-7140f1985c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx\" (UID: \"b056e2ac-15c4-4c5e-850e-7140f1985c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"
Sep 10 21:31:17.676965 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.676948    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b056e2ac-15c4-4c5e-850e-7140f1985c27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx\" (UID: \"b056e2ac-15c4-4c5e-850e-7140f1985c27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"
Sep 10 21:31:17.840129 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:17.840042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"
Sep 10 21:31:18.110005 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:18.109980    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"]
Sep 10 21:31:18.112629 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:31:18.112601    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb056e2ac_15c4_4c5e_850e_7140f1985c27.slice/crio-22715c0acef6033f0a7899790da890aa1f2255fd3d3abb6d5fd7733e5d0632a2 WatchSource:0}: Error finding container 22715c0acef6033f0a7899790da890aa1f2255fd3d3abb6d5fd7733e5d0632a2: Status 404 returned error can't find the container with id 22715c0acef6033f0a7899790da890aa1f2255fd3d3abb6d5fd7733e5d0632a2
Sep 10 21:31:19.058166 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:19.058134    2583 generic.go:356] "Generic (PLEG): container finished" podID="b056e2ac-15c4-4c5e-850e-7140f1985c27" containerID="4fc5b6ce9fabafebc4e7a3ac7cf0283c2a66b34dad78613240e87edd8897f0d2" exitCode=0
Sep 10 21:31:19.058597 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:19.058217    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx" event={"ID":"b056e2ac-15c4-4c5e-850e-7140f1985c27","Type":"ContainerDied","Data":"4fc5b6ce9fabafebc4e7a3ac7cf0283c2a66b34dad78613240e87edd8897f0d2"}
Sep 10 21:31:19.058597 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:19.058271    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx" event={"ID":"b056e2ac-15c4-4c5e-850e-7140f1985c27","Type":"ContainerStarted","Data":"22715c0acef6033f0a7899790da890aa1f2255fd3d3abb6d5fd7733e5d0632a2"}
Sep 10 21:31:20.199261 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:20.199237    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"
Sep 10 21:31:20.305177 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:20.305141    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b056e2ac-15c4-4c5e-850e-7140f1985c27-maas-api-service-ca\") pod \"b056e2ac-15c4-4c5e-850e-7140f1985c27\" (UID: \"b056e2ac-15c4-4c5e-850e-7140f1985c27\") "
Sep 10 21:31:20.305522 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:20.305483    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b056e2ac-15c4-4c5e-850e-7140f1985c27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b056e2ac-15c4-4c5e-850e-7140f1985c27" (UID: "b056e2ac-15c4-4c5e-850e-7140f1985c27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:31:20.406603 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:20.406568    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b056e2ac-15c4-4c5e-850e-7140f1985c27-maas-api-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:31:21.069813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:21.069777    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx" event={"ID":"b056e2ac-15c4-4c5e-850e-7140f1985c27","Type":"ContainerDied","Data":"22715c0acef6033f0a7899790da890aa1f2255fd3d3abb6d5fd7733e5d0632a2"}
Sep 10 21:31:21.069813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:21.069797    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"
Sep 10 21:31:21.070026 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:21.069827    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22715c0acef6033f0a7899790da890aa1f2255fd3d3abb6d5fd7733e5d0632a2"
Sep 10 21:31:24.347059 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:24.346994    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:24.447563 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:24.447530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:33.156296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.156259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:33.191203 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.191166    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"]
Sep 10 21:31:33.191472 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.191438    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4" podUID="ce05c1e9-ad62-4672-83a4-2234da27c068" containerName="payload-processing" containerID="cri-o://570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7" gracePeriod=30
Sep 10 21:31:33.203068 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.202497    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"]
Sep 10 21:31:33.203068 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.202729    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9" podUID="8facb9f1-7c89-4986-a398-cc416c118612" containerName="payload-pre-processing" containerID="cri-o://00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208" gracePeriod=30
Sep 10 21:31:33.233336 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.233303    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:33.296688 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.296649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:33.391538 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.391494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:33.403670 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.403636    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"]
Sep 10 21:31:33.405696 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.405669    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7ce89dd6-nknlx"]
Sep 10 21:31:33.451199 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.451163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:33.487491 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.487466    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:31:33.530240 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.529044    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:31:33.632549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.632451    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce05c1e9-ad62-4672-83a4-2234da27c068-plugins-config-volume\") pod \"ce05c1e9-ad62-4672-83a4-2234da27c068\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") "
Sep 10 21:31:33.632757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.632592    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6lnp\" (UniqueName: \"kubernetes.io/projected/ce05c1e9-ad62-4672-83a4-2234da27c068-kube-api-access-t6lnp\") pod \"ce05c1e9-ad62-4672-83a4-2234da27c068\" (UID: \"ce05c1e9-ad62-4672-83a4-2234da27c068\") "
Sep 10 21:31:33.632757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.632695    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8facb9f1-7c89-4986-a398-cc416c118612-plugins-config-volume\") pod \"8facb9f1-7c89-4986-a398-cc416c118612\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") "
Sep 10 21:31:33.632757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.632730    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcsmx\" (UniqueName: \"kubernetes.io/projected/8facb9f1-7c89-4986-a398-cc416c118612-kube-api-access-lcsmx\") pod \"8facb9f1-7c89-4986-a398-cc416c118612\" (UID: \"8facb9f1-7c89-4986-a398-cc416c118612\") "
Sep 10 21:31:33.633089 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.632946    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce05c1e9-ad62-4672-83a4-2234da27c068-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce05c1e9-ad62-4672-83a4-2234da27c068" (UID: "ce05c1e9-ad62-4672-83a4-2234da27c068"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:31:33.633172 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.633120    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8facb9f1-7c89-4986-a398-cc416c118612-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8facb9f1-7c89-4986-a398-cc416c118612" (UID: "8facb9f1-7c89-4986-a398-cc416c118612"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:31:33.634766 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.634738    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce05c1e9-ad62-4672-83a4-2234da27c068-kube-api-access-t6lnp" (OuterVolumeSpecName: "kube-api-access-t6lnp") pod "ce05c1e9-ad62-4672-83a4-2234da27c068" (UID: "ce05c1e9-ad62-4672-83a4-2234da27c068"). InnerVolumeSpecName "kube-api-access-t6lnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:31:33.634852 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.634797    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8facb9f1-7c89-4986-a398-cc416c118612-kube-api-access-lcsmx" (OuterVolumeSpecName: "kube-api-access-lcsmx") pod "8facb9f1-7c89-4986-a398-cc416c118612" (UID: "8facb9f1-7c89-4986-a398-cc416c118612"). InnerVolumeSpecName "kube-api-access-lcsmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:31:33.734639 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.734549    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8facb9f1-7c89-4986-a398-cc416c118612-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:31:33.734639 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.734583    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcsmx\" (UniqueName: \"kubernetes.io/projected/8facb9f1-7c89-4986-a398-cc416c118612-kube-api-access-lcsmx\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:31:33.734639 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.734593    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce05c1e9-ad62-4672-83a4-2234da27c068-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:31:33.734639 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.734602    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6lnp\" (UniqueName: \"kubernetes.io/projected/ce05c1e9-ad62-4672-83a4-2234da27c068-kube-api-access-t6lnp\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:31:33.960673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:33.960643    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b056e2ac-15c4-4c5e-850e-7140f1985c27" path="/var/lib/kubelet/pods/b056e2ac-15c4-4c5e-850e-7140f1985c27/volumes"
Sep 10 21:31:34.142163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.142131    2583 generic.go:356] "Generic (PLEG): container finished" podID="8facb9f1-7c89-4986-a398-cc416c118612" containerID="00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208" exitCode=0
Sep 10 21:31:34.142334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.142178    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"
Sep 10 21:31:34.142334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.142216    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9" event={"ID":"8facb9f1-7c89-4986-a398-cc416c118612","Type":"ContainerDied","Data":"00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208"}
Sep 10 21:31:34.142334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.142268    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9" event={"ID":"8facb9f1-7c89-4986-a398-cc416c118612","Type":"ContainerDied","Data":"ac1e004cd5f259a28319be1e8cd524449168845bdc056d0090a115366b2f7b98"}
Sep 10 21:31:34.142334 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.142299    2583 scope.go:117] "RemoveContainer" containerID="00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208"
Sep 10 21:31:34.143684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.143658    2583 generic.go:356] "Generic (PLEG): container finished" podID="ce05c1e9-ad62-4672-83a4-2234da27c068" containerID="570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7" exitCode=0
Sep 10 21:31:34.143792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.143702    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"
Sep 10 21:31:34.143792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.143744    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4" event={"ID":"ce05c1e9-ad62-4672-83a4-2234da27c068","Type":"ContainerDied","Data":"570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7"}
Sep 10 21:31:34.143792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.143781    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4" event={"ID":"ce05c1e9-ad62-4672-83a4-2234da27c068","Type":"ContainerDied","Data":"01d5f12a8734282d5916d3677a78e506b027c72e3d51b7957591754a212e663c"}
Sep 10 21:31:34.151045 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.151028    2583 scope.go:117] "RemoveContainer" containerID="00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208"
Sep 10 21:31:34.151284 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:31:34.151267    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208\": container with ID starting with 00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208 not found: ID does not exist" containerID="00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208"
Sep 10 21:31:34.151323 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.151293    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208"} err="failed to get container status \"00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208\": rpc error: code = NotFound desc = could not find container \"00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208\": container with ID starting with 00a14d3f3891df2d5af1c92efc1005eecde362236a4c7a4d820f07c27b847208 not found: ID does not exist"
Sep 10 21:31:34.151323 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.151308    2583 scope.go:117] "RemoveContainer" containerID="570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7"
Sep 10 21:31:34.159834 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.159812    2583 scope.go:117] "RemoveContainer" containerID="570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7"
Sep 10 21:31:34.160144 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:31:34.160126    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7\": container with ID starting with 570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7 not found: ID does not exist" containerID="570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7"
Sep 10 21:31:34.160200 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.160159    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7"} err="failed to get container status \"570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7\": rpc error: code = NotFound desc = could not find container \"570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7\": container with ID starting with 570ff573005302ef9255582a7ca7b9d9fdae793491d5fea4e766f9c8be94e2e7 not found: ID does not exist"
Sep 10 21:31:34.160259 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.160243    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"]
Sep 10 21:31:34.164600 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.164574    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7ce89dd6-864cdd8d45-nwpp4"]
Sep 10 21:31:34.175357 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.175334    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"]
Sep 10 21:31:34.182081 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:34.182059    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7ce89dd6-7fcc45c5cd-h5ql9"]
Sep 10 21:31:35.960639 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:35.960608    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8facb9f1-7c89-4986-a398-cc416c118612" path="/var/lib/kubelet/pods/8facb9f1-7c89-4986-a398-cc416c118612/volumes"
Sep 10 21:31:35.961033 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:35.960965    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce05c1e9-ad62-4672-83a4-2234da27c068" path="/var/lib/kubelet/pods/ce05c1e9-ad62-4672-83a4-2234da27c068/volumes"
Sep 10 21:31:36.442592 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:36.442555    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:41.480939 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:41.480899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:51.642923 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:51.642887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:54.059729 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:54.059694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:54.125913 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:54.125878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:59.035547 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:59.035512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:59.445645 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:31:59.445606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:05.101648 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:05.101603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:10.961472 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:10.961438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:11.064331 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:11.064299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:11.228362 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:11.228273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:12.441247 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:12.441215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:12.533348 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:12.533312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:12.633864 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:12.633832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:13.219373 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:13.219341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:13.314472 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:13.314434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:13.411842 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:13.411801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:14.619235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:14.619195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:20.413003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:20.412965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:20.517102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:20.517074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:21.068114 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:21.068074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:27.047239 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:27.047191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:27.117827 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:27.117792    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:27.201480 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:27.201442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:31.931630 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:31.931591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:32.932674 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:32.932631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:37.737981 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:37.737943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:37.951903 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:37.951871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:43.739272 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:43.739234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:51.485568 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:51.485494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:58.842768 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:32:58.842736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:00.679174 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:00.679131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:04.548518 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:04.548467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:04.611075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:04.611041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:04.905493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:04.905458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:05.232197 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:05.232102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:05.334634 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:05.334601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:08.163152 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:08.163110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:08.260821 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:08.260789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:09.519106 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:09.519069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:09.621195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:09.621159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:09.823396 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:09.823304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:10.082416 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.082330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:10.183810 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.183769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:10.216880 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.216850    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"]
Sep 10 21:33:10.217264 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217252    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b056e2ac-15c4-4c5e-850e-7140f1985c27" containerName="revoke-keys"
Sep 10 21:33:10.217311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217265    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b056e2ac-15c4-4c5e-850e-7140f1985c27" containerName="revoke-keys"
Sep 10 21:33:10.217311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217274    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8facb9f1-7c89-4986-a398-cc416c118612" containerName="payload-pre-processing"
Sep 10 21:33:10.217311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217280    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8facb9f1-7c89-4986-a398-cc416c118612" containerName="payload-pre-processing"
Sep 10 21:33:10.217311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217296    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce05c1e9-ad62-4672-83a4-2234da27c068" containerName="payload-processing"
Sep 10 21:33:10.217311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217302    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce05c1e9-ad62-4672-83a4-2234da27c068" containerName="payload-processing"
Sep 10 21:33:10.217471 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217367    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8facb9f1-7c89-4986-a398-cc416c118612" containerName="payload-pre-processing"
Sep 10 21:33:10.217471 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217376    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b056e2ac-15c4-4c5e-850e-7140f1985c27" containerName="revoke-keys"
Sep 10 21:33:10.217471 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217383    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="381bf1d3-7259-48c7-8b9f-dc73802a5c38" containerName="cleanup"
Sep 10 21:33:10.217471 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.217390    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce05c1e9-ad62-4672-83a4-2234da27c068" containerName="payload-processing"
Sep 10 21:33:10.220344 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.220326    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.222852 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.222825    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-77a957-dockercfg-l7qtw\""
Sep 10 21:33:10.222979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.222831    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-77a957\""
Sep 10 21:33:10.231363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.231342    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"]
Sep 10 21:33:10.272912 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.272881    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"]
Sep 10 21:33:10.276777 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.276757    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.288281 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.288256    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"]
Sep 10 21:33:10.359895 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.359809    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7bcb36f-157e-4702-ae34-aa3c6d2be354-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.359895 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.359873    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6qb2\" (UniqueName: \"kubernetes.io/projected/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-kube-api-access-t6qb2\") pod \"payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") " pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.360086 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.359921    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkxmm\" (UniqueName: \"kubernetes.io/projected/e7bcb36f-157e-4702-ae34-aa3c6d2be354-kube-api-access-jkxmm\") pod \"payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.360086 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.360033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-plugins-config-volume\") pod \"payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") " pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.461393 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.461356    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6qb2\" (UniqueName: \"kubernetes.io/projected/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-kube-api-access-t6qb2\") pod \"payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") " pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.461621 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.461426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkxmm\" (UniqueName: \"kubernetes.io/projected/e7bcb36f-157e-4702-ae34-aa3c6d2be354-kube-api-access-jkxmm\") pod \"payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.461621 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.461556    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-plugins-config-volume\") pod \"payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") " pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.461621 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.461604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7bcb36f-157e-4702-ae34-aa3c6d2be354-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.462363 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.462343    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7bcb36f-157e-4702-ae34-aa3c6d2be354-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.462429 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.462373    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-plugins-config-volume\") pod \"payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") " pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.468982 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.468951    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkxmm\" (UniqueName: \"kubernetes.io/projected/e7bcb36f-157e-4702-ae34-aa3c6d2be354-kube-api-access-jkxmm\") pod \"payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.469094 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.468994    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6qb2\" (UniqueName: \"kubernetes.io/projected/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-kube-api-access-t6qb2\") pod \"payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") " pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.531782 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.531751    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:10.587902 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.587846    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:10.804814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.804788    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"]
Sep 10 21:33:10.806744 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:33:10.806718    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e3bfd2_aee7_4e1d_95cd_20cb7ab95b67.slice/crio-4348ceeeec1b261c4284ec99e6bcdf3abf53e45b088680418ce14877b9735285 WatchSource:0}: Error finding container 4348ceeeec1b261c4284ec99e6bcdf3abf53e45b088680418ce14877b9735285: Status 404 returned error can't find the container with id 4348ceeeec1b261c4284ec99e6bcdf3abf53e45b088680418ce14877b9735285
Sep 10 21:33:10.872263 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:10.872203    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"]
Sep 10 21:33:10.875889 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:33:10.875865    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7bcb36f_157e_4702_ae34_aa3c6d2be354.slice/crio-c5f3334e184643ee53f36caaea336fe3bf06c112a12681b18c5a7ffe57965503 WatchSource:0}: Error finding container c5f3334e184643ee53f36caaea336fe3bf06c112a12681b18c5a7ffe57965503: Status 404 returned error can't find the container with id c5f3334e184643ee53f36caaea336fe3bf06c112a12681b18c5a7ffe57965503
Sep 10 21:33:11.661056 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.661013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf" event={"ID":"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67","Type":"ContainerStarted","Data":"ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e"}
Sep 10 21:33:11.661557 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.661082    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf" event={"ID":"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67","Type":"ContainerStarted","Data":"4348ceeeec1b261c4284ec99e6bcdf3abf53e45b088680418ce14877b9735285"}
Sep 10 21:33:11.661557 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.661207    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:11.662659 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.662634    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq" event={"ID":"e7bcb36f-157e-4702-ae34-aa3c6d2be354","Type":"ContainerStarted","Data":"11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2"}
Sep 10 21:33:11.662782 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.662673    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq" event={"ID":"e7bcb36f-157e-4702-ae34-aa3c6d2be354","Type":"ContainerStarted","Data":"c5f3334e184643ee53f36caaea336fe3bf06c112a12681b18c5a7ffe57965503"}
Sep 10 21:33:11.662782 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.662758    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:11.680543 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.680469    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf" podStartSLOduration=1.680446858 podStartE2EDuration="1.680446858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:11.677757434 +0000 UTC m=+1382.204721779" watchObservedRunningTime="2026-09-10 21:33:11.680446858 +0000 UTC m=+1382.207411207"
Sep 10 21:33:11.695739 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:11.695686    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq" podStartSLOduration=1.695666525 podStartE2EDuration="1.695666525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:11.693094104 +0000 UTC m=+1382.220058451" watchObservedRunningTime="2026-09-10 21:33:11.695666525 +0000 UTC m=+1382.222630875"
Sep 10 21:33:18.534057 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:18.534015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:22.670953 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:22.670921    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:33:22.671341 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:22.670995    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:33:33.062016 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:33.061978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:33.478522 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:33.478465    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:33.797585 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:33.797479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:34.747384 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:34.747350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:37.270456 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:37.270423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:37.368453 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:37.368417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:38.442496 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.442464    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"]
Sep 10 21:33:38.447664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.447642    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.449701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.449678    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c71bd0-serving-cert\""
Sep 10 21:33:38.455013 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.454968    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"]
Sep 10 21:33:38.479923 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.479890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:38.535690 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.535650    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e070697d-01b7-41c4-a91b-16e25fcb9997-openshift-service-ca\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.535876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.535705    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhvp9\" (UniqueName: \"kubernetes.io/projected/e070697d-01b7-41c4-a91b-16e25fcb9997-kube-api-access-vhvp9\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.535876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.535731    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-metrics-certs\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.535876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.535805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.585066 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.585032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:38.637092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.637053    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhvp9\" (UniqueName: \"kubernetes.io/projected/e070697d-01b7-41c4-a91b-16e25fcb9997-kube-api-access-vhvp9\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.637295 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.637220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-metrics-certs\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.637367 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.637302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.637452 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.637423    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e070697d-01b7-41c4-a91b-16e25fcb9997-openshift-service-ca\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.637536 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:33:38.637459    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-serving-cert: secret "maas-api-e2e-api-b-c71bd0-serving-cert" not found
Sep 10 21:33:38.637605 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:33:38.637561    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls podName:e070697d-01b7-41c4-a91b-16e25fcb9997 nodeName:}" failed. No retries permitted until 2026-09-10 21:33:39.137539765 +0000 UTC m=+1409.664504104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls") pod "maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt" (UID: "e070697d-01b7-41c4-a91b-16e25fcb9997") : secret "maas-api-e2e-api-b-c71bd0-serving-cert" not found
Sep 10 21:33:38.638197 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.638164    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e070697d-01b7-41c4-a91b-16e25fcb9997-openshift-service-ca\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.640067 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.640041    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-metrics-certs\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.646120 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.646092    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhvp9\" (UniqueName: \"kubernetes.io/projected/e070697d-01b7-41c4-a91b-16e25fcb9997-kube-api-access-vhvp9\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:38.801356 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:38.801250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:39.142217 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.142190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:39.144586 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.144562    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls\") pod \"maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:39.146397 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.146360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:39.242349 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.242313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:39.364409 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.364370    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:39.653619 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.653537    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"]
Sep 10 21:33:39.656617 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:33:39.656578    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode070697d_01b7_41c4_a91b_16e25fcb9997.slice/crio-0b2679927f47cdfb005abfff0eaebe49108fdca9d25c678a43137a586c9f975d WatchSource:0}: Error finding container 0b2679927f47cdfb005abfff0eaebe49108fdca9d25c678a43137a586c9f975d: Status 404 returned error can't find the container with id 0b2679927f47cdfb005abfff0eaebe49108fdca9d25c678a43137a586c9f975d
Sep 10 21:33:39.658036 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.658020    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:33:39.812941 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:39.812862    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt" event={"ID":"e070697d-01b7-41c4-a91b-16e25fcb9997","Type":"ContainerStarted","Data":"0b2679927f47cdfb005abfff0eaebe49108fdca9d25c678a43137a586c9f975d"}
Sep 10 21:33:40.820163 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:40.820118    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt" event={"ID":"e070697d-01b7-41c4-a91b-16e25fcb9997","Type":"ContainerStarted","Data":"1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979"}
Sep 10 21:33:40.820578 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:40.820227    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:40.841797 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:40.841731    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt" podStartSLOduration=2.484093502 podStartE2EDuration="2.841711409s" totalImagesPullingTime="357.617907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:40.840880021 +0000 UTC m=+1411.367844382" watchObservedRunningTime="2026-09-10 21:33:40.841711409 +0000 UTC m=+1411.368675757"
Sep 10 21:33:46.831541 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:46.831490    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:33:47.016679 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:47.016645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:48.011553 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:48.011521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:54.688024 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:54.687990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:55.854545 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:55.854498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:59.589309 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:33:59.589273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:02.197858 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:02.197781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:05.100615 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:05.100550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:14.721947 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:14.721892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:17.754853 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:17.754819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.375930 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.375890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.401933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.401901    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"]
Sep 10 21:34:20.402172 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.402144    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf" podUID="28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" containerName="payload-processing" containerID="cri-o://ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e" gracePeriod=30
Sep 10 21:34:20.411179 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.411155    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"]
Sep 10 21:34:20.411354 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.411332    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq" podUID="e7bcb36f-157e-4702-ae34-aa3c6d2be354" containerName="payload-pre-processing" containerID="cri-o://11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2" gracePeriod=30
Sep 10 21:34:20.444972 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.444939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.510814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.510782    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.611017 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.610979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.671904 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.671877    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:34:20.692923 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.692543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.704659 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.704633    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:34:20.744622 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.743333    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-plugins-config-volume\") pod \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") "
Sep 10 21:34:20.744622 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.743483    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6qb2\" (UniqueName: \"kubernetes.io/projected/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-kube-api-access-t6qb2\") pod \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\" (UID: \"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67\") "
Sep 10 21:34:20.745070 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.745043    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" (UID: "28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:20.747196 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.747151    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-kube-api-access-t6qb2" (OuterVolumeSpecName: "kube-api-access-t6qb2") pod "28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" (UID: "28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67"). InnerVolumeSpecName "kube-api-access-t6qb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:20.844661 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.844625    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkxmm\" (UniqueName: \"kubernetes.io/projected/e7bcb36f-157e-4702-ae34-aa3c6d2be354-kube-api-access-jkxmm\") pod \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") "
Sep 10 21:34:20.844848 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.844727    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7bcb36f-157e-4702-ae34-aa3c6d2be354-plugins-config-volume\") pod \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\" (UID: \"e7bcb36f-157e-4702-ae34-aa3c6d2be354\") "
Sep 10 21:34:20.845001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.844974    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6qb2\" (UniqueName: \"kubernetes.io/projected/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-kube-api-access-t6qb2\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:20.845001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.844988    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:20.845173 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.845099    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7bcb36f-157e-4702-ae34-aa3c6d2be354-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e7bcb36f-157e-4702-ae34-aa3c6d2be354" (UID: "e7bcb36f-157e-4702-ae34-aa3c6d2be354"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:20.846826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.846802    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7bcb36f-157e-4702-ae34-aa3c6d2be354-kube-api-access-jkxmm" (OuterVolumeSpecName: "kube-api-access-jkxmm") pod "e7bcb36f-157e-4702-ae34-aa3c6d2be354" (UID: "e7bcb36f-157e-4702-ae34-aa3c6d2be354"). InnerVolumeSpecName "kube-api-access-jkxmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:20.942079 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.941993    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.946236 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.946215    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7bcb36f-157e-4702-ae34-aa3c6d2be354-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:20.946308 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:20.946238    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkxmm\" (UniqueName: \"kubernetes.io/projected/e7bcb36f-157e-4702-ae34-aa3c6d2be354-kube-api-access-jkxmm\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:21.036785 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.036747    2583 generic.go:356] "Generic (PLEG): container finished" podID="28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" containerID="ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e" exitCode=0
Sep 10 21:34:21.036933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.036794    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"
Sep 10 21:34:21.036933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.036836    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf" event={"ID":"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67","Type":"ContainerDied","Data":"ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e"}
Sep 10 21:34:21.036933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.036890    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf" event={"ID":"28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67","Type":"ContainerDied","Data":"4348ceeeec1b261c4284ec99e6bcdf3abf53e45b088680418ce14877b9735285"}
Sep 10 21:34:21.036933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.036914    2583 scope.go:117] "RemoveContainer" containerID="ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e"
Sep 10 21:34:21.038266 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.038246    2583 generic.go:356] "Generic (PLEG): container finished" podID="e7bcb36f-157e-4702-ae34-aa3c6d2be354" containerID="11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2" exitCode=0
Sep 10 21:34:21.038361 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.038300    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"
Sep 10 21:34:21.038361 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.038330    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq" event={"ID":"e7bcb36f-157e-4702-ae34-aa3c6d2be354","Type":"ContainerDied","Data":"11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2"}
Sep 10 21:34:21.038478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.038371    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq" event={"ID":"e7bcb36f-157e-4702-ae34-aa3c6d2be354","Type":"ContainerDied","Data":"c5f3334e184643ee53f36caaea336fe3bf06c112a12681b18c5a7ffe57965503"}
Sep 10 21:34:21.045990 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.045967    2583 scope.go:117] "RemoveContainer" containerID="ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e"
Sep 10 21:34:21.046354 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:34:21.046323    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e\": container with ID starting with ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e not found: ID does not exist" containerID="ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e"
Sep 10 21:34:21.046460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.046365    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e"} err="failed to get container status \"ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e\": rpc error: code = NotFound desc = could not find container \"ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e\": container with ID starting with ebe290903f6e2c3c26f6117b5836b113ceb7a2921061368d2e936cdc5befd89e not found: ID does not exist"
Sep 10 21:34:21.046460 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.046387    2583 scope.go:117] "RemoveContainer" containerID="11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2"
Sep 10 21:34:21.055047 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.055029    2583 scope.go:117] "RemoveContainer" containerID="11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2"
Sep 10 21:34:21.055324 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:34:21.055305    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2\": container with ID starting with 11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2 not found: ID does not exist" containerID="11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2"
Sep 10 21:34:21.055403 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.055331    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2"} err="failed to get container status \"11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2\": rpc error: code = NotFound desc = could not find container \"11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2\": container with ID starting with 11a8792243f6f65812f60f9d500aed44c63642882e0b58e50ed68bc11d7724e2 not found: ID does not exist"
Sep 10 21:34:21.063071 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.063047    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"]
Sep 10 21:34:21.066430 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.066404    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-77a957-5b8d446974-8m7wq"]
Sep 10 21:34:21.076426 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.076208    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"]
Sep 10 21:34:21.078399 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.078375    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-77a957-f747b9dc5-d6gnf"]
Sep 10 21:34:21.884113 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.884012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:21.960406 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.960377    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" path="/var/lib/kubelet/pods/28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67/volumes"
Sep 10 21:34:21.960780 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:21.960766    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7bcb36f-157e-4702-ae34-aa3c6d2be354" path="/var/lib/kubelet/pods/e7bcb36f-157e-4702-ae34-aa3c6d2be354/volumes"
Sep 10 21:34:26.786554 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:26.786516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:26.886311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:26.886275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:27.040103 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:27.040010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:27.493599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:27.493556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:42.890972 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:42.890937    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"]
Sep 10 21:34:42.891995 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:42.891962    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt" podUID="e070697d-01b7-41c4-a91b-16e25fcb9997" containerName="maas-api" containerID="cri-o://1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979" gracePeriod=30
Sep 10 21:34:42.920586 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:42.920541    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:43.007732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.007700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:43.079794 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.079767    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:43.148580 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.148520    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:34:43.157058 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.157029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:43.157195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.157057    2583 generic.go:356] "Generic (PLEG): container finished" podID="e070697d-01b7-41c4-a91b-16e25fcb9997" containerID="1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979" exitCode=0
Sep 10 21:34:43.157195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.157101    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"
Sep 10 21:34:43.157195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.157114    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt" event={"ID":"e070697d-01b7-41c4-a91b-16e25fcb9997","Type":"ContainerDied","Data":"1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979"}
Sep 10 21:34:43.157195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.157168    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt" event={"ID":"e070697d-01b7-41c4-a91b-16e25fcb9997","Type":"ContainerDied","Data":"0b2679927f47cdfb005abfff0eaebe49108fdca9d25c678a43137a586c9f975d"}
Sep 10 21:34:43.157342 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.157197    2583 scope.go:117] "RemoveContainer" containerID="1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979"
Sep 10 21:34:43.165703 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.165684    2583 scope.go:117] "RemoveContainer" containerID="1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979"
Sep 10 21:34:43.165961 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:34:43.165944    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979\": container with ID starting with 1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979 not found: ID does not exist" containerID="1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979"
Sep 10 21:34:43.166019 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.165971    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979"} err="failed to get container status \"1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979\": rpc error: code = NotFound desc = could not find container \"1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979\": container with ID starting with 1165eb6fc2a535023e484be5a7712a8b40473c1933f045061fc6e6bcd866c979 not found: ID does not exist"
Sep 10 21:34:43.243155 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.243122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:43.269517 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.269472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls\") pod \"e070697d-01b7-41c4-a91b-16e25fcb9997\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") "
Sep 10 21:34:43.269691 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.269548    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhvp9\" (UniqueName: \"kubernetes.io/projected/e070697d-01b7-41c4-a91b-16e25fcb9997-kube-api-access-vhvp9\") pod \"e070697d-01b7-41c4-a91b-16e25fcb9997\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") "
Sep 10 21:34:43.269691 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.269644    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e070697d-01b7-41c4-a91b-16e25fcb9997-openshift-service-ca\") pod \"e070697d-01b7-41c4-a91b-16e25fcb9997\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") "
Sep 10 21:34:43.269691 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.269689    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-metrics-certs\") pod \"e070697d-01b7-41c4-a91b-16e25fcb9997\" (UID: \"e070697d-01b7-41c4-a91b-16e25fcb9997\") "
Sep 10 21:34:43.270362 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.270329    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e070697d-01b7-41c4-a91b-16e25fcb9997-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e070697d-01b7-41c4-a91b-16e25fcb9997" (UID: "e070697d-01b7-41c4-a91b-16e25fcb9997"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:43.271768 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.271733    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e070697d-01b7-41c4-a91b-16e25fcb9997" (UID: "e070697d-01b7-41c4-a91b-16e25fcb9997"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:43.271885 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.271868    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e070697d-01b7-41c4-a91b-16e25fcb9997" (UID: "e070697d-01b7-41c4-a91b-16e25fcb9997"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:43.272203 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.272172    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e070697d-01b7-41c4-a91b-16e25fcb9997-kube-api-access-vhvp9" (OuterVolumeSpecName: "kube-api-access-vhvp9") pod "e070697d-01b7-41c4-a91b-16e25fcb9997" (UID: "e070697d-01b7-41c4-a91b-16e25fcb9997"). InnerVolumeSpecName "kube-api-access-vhvp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:43.371175 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.371135    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-maas-api-tls\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:43.371175 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.371167    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhvp9\" (UniqueName: \"kubernetes.io/projected/e070697d-01b7-41c4-a91b-16e25fcb9997-kube-api-access-vhvp9\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:43.371175 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.371181    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e070697d-01b7-41c4-a91b-16e25fcb9997-openshift-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:43.371415 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.371193    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e070697d-01b7-41c4-a91b-16e25fcb9997-metrics-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:43.478731 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.478701    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"]
Sep 10 21:34:43.482774 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.482743    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c71bd0-cf68c548f-s8jdt"]
Sep 10 21:34:43.525285 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.525255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:43.625691 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.625657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:43.960426 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:43.960390    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e070697d-01b7-41c4-a91b-16e25fcb9997" path="/var/lib/kubelet/pods/e070697d-01b7-41c4-a91b-16e25fcb9997/volumes"
Sep 10 21:34:49.440672 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.440628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:49.514856 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.514826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:49.691901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.691814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:49.999471 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.999379    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"]
Sep 10 21:34:49.999905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.999886    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" containerName="payload-processing"
Sep 10 21:34:49.999997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.999908    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" containerName="payload-processing"
Sep 10 21:34:49.999997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.999933    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e070697d-01b7-41c4-a91b-16e25fcb9997" containerName="maas-api"
Sep 10 21:34:49.999997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.999942    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e070697d-01b7-41c4-a91b-16e25fcb9997" containerName="maas-api"
Sep 10 21:34:49.999997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.999962    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7bcb36f-157e-4702-ae34-aa3c6d2be354" containerName="payload-pre-processing"
Sep 10 21:34:49.999997 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:49.999970    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7bcb36f-157e-4702-ae34-aa3c6d2be354" containerName="payload-pre-processing"
Sep 10 21:34:50.000259 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.000053    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7bcb36f-157e-4702-ae34-aa3c6d2be354" containerName="payload-pre-processing"
Sep 10 21:34:50.000259 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.000068    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e3bfd2-aee7-4e1d-95cd-20cb7ab95b67" containerName="payload-processing"
Sep 10 21:34:50.000259 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.000081    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e070697d-01b7-41c4-a91b-16e25fcb9997" containerName="maas-api"
Sep 10 21:34:50.004983 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.004958    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"
Sep 10 21:34:50.006963 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.006940    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:34:50.009850 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.009826    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"]
Sep 10 21:34:50.120436 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.120403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:50.141496 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.141462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680f7e-db2d-4b11-b45d-1d2299ffc258-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h\" (UID: \"3f680f7e-db2d-4b11-b45d-1d2299ffc258\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"
Sep 10 21:34:50.242807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.242774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680f7e-db2d-4b11-b45d-1d2299ffc258-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h\" (UID: \"3f680f7e-db2d-4b11-b45d-1d2299ffc258\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"
Sep 10 21:34:50.243408 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.243390    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680f7e-db2d-4b11-b45d-1d2299ffc258-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h\" (UID: \"3f680f7e-db2d-4b11-b45d-1d2299ffc258\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"
Sep 10 21:34:50.316684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.316594    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"
Sep 10 21:34:50.613799 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:50.613776    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"]
Sep 10 21:34:50.616332 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:34:50.616302    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f680f7e_db2d_4b11_b45d_1d2299ffc258.slice/crio-a44367baa3b633819ef16795b13bf47b2189890f3b5a6023af5df3444661f5c0 WatchSource:0}: Error finding container a44367baa3b633819ef16795b13bf47b2189890f3b5a6023af5df3444661f5c0: Status 404 returned error can't find the container with id a44367baa3b633819ef16795b13bf47b2189890f3b5a6023af5df3444661f5c0
Sep 10 21:34:51.205855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:51.205761    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h" event={"ID":"3f680f7e-db2d-4b11-b45d-1d2299ffc258","Type":"ContainerStarted","Data":"4fae12d55d24abc1c221028da21ce65cf48e2946e68e5175dc655e678a4b402a"}
Sep 10 21:34:51.205855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:51.205821    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h" event={"ID":"3f680f7e-db2d-4b11-b45d-1d2299ffc258","Type":"ContainerStarted","Data":"a44367baa3b633819ef16795b13bf47b2189890f3b5a6023af5df3444661f5c0"}
Sep 10 21:34:51.220439 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:51.220371    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h" podStartSLOduration=2.220348732 podStartE2EDuration="2.220348732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:51.21921766 +0000 UTC m=+1481.746182001" watchObservedRunningTime="2026-09-10 21:34:51.220348732 +0000 UTC m=+1481.747313080"
Sep 10 21:34:52.212403 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:52.212371    2583 generic.go:356] "Generic (PLEG): container finished" podID="3f680f7e-db2d-4b11-b45d-1d2299ffc258" containerID="4fae12d55d24abc1c221028da21ce65cf48e2946e68e5175dc655e678a4b402a" exitCode=0
Sep 10 21:34:52.212820 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:52.212425    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h" event={"ID":"3f680f7e-db2d-4b11-b45d-1d2299ffc258","Type":"ContainerDied","Data":"4fae12d55d24abc1c221028da21ce65cf48e2946e68e5175dc655e678a4b402a"}
Sep 10 21:34:53.353941 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:53.353910    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"
Sep 10 21:34:53.474131 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:53.474033    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680f7e-db2d-4b11-b45d-1d2299ffc258-maas-api-service-ca\") pod \"3f680f7e-db2d-4b11-b45d-1d2299ffc258\" (UID: \"3f680f7e-db2d-4b11-b45d-1d2299ffc258\") "
Sep 10 21:34:53.474357 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:53.474336    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f680f7e-db2d-4b11-b45d-1d2299ffc258-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f680f7e-db2d-4b11-b45d-1d2299ffc258" (UID: "3f680f7e-db2d-4b11-b45d-1d2299ffc258"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:53.575237 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:53.575199    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f680f7e-db2d-4b11-b45d-1d2299ffc258-maas-api-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:34:54.224991 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:54.224915    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"
Sep 10 21:34:54.225143 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:54.224912    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h" event={"ID":"3f680f7e-db2d-4b11-b45d-1d2299ffc258","Type":"ContainerDied","Data":"a44367baa3b633819ef16795b13bf47b2189890f3b5a6023af5df3444661f5c0"}
Sep 10 21:34:54.225143 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:34:54.225032    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a44367baa3b633819ef16795b13bf47b2189890f3b5a6023af5df3444661f5c0"
Sep 10 21:35:00.547603 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.547561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:00.584223 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.584183    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"]
Sep 10 21:35:00.584468 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.584442    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz" podUID="881f31d8-d421-48d0-9978-9ea26f413b44" containerName="payload-processing" containerID="cri-o://fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9" gracePeriod=30
Sep 10 21:35:00.591132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.591033    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"]
Sep 10 21:35:00.591523 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.591287    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6" podUID="8a92b351-087e-4c66-8660-c0df76ec3ba7" containerName="payload-pre-processing" containerID="cri-o://9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697" gracePeriod=30
Sep 10 21:35:00.630710 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.630662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:00.699905 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.699873    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:00.779952 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.779918    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"]
Sep 10 21:35:00.782701 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.782677    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bd7b8a-mzc2h"]
Sep 10 21:35:00.786032 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.786006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:00.849126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.848892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:00.873095 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.873069    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:35:00.880654 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.880631    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:35:00.943224 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.943194    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7wpd\" (UniqueName: \"kubernetes.io/projected/881f31d8-d421-48d0-9978-9ea26f413b44-kube-api-access-m7wpd\") pod \"881f31d8-d421-48d0-9978-9ea26f413b44\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") "
Sep 10 21:35:00.943405 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.943287    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881f31d8-d421-48d0-9978-9ea26f413b44-plugins-config-volume\") pod \"881f31d8-d421-48d0-9978-9ea26f413b44\" (UID: \"881f31d8-d421-48d0-9978-9ea26f413b44\") "
Sep 10 21:35:00.943704 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.943679    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881f31d8-d421-48d0-9978-9ea26f413b44-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "881f31d8-d421-48d0-9978-9ea26f413b44" (UID: "881f31d8-d421-48d0-9978-9ea26f413b44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:00.945298 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:00.945273    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881f31d8-d421-48d0-9978-9ea26f413b44-kube-api-access-m7wpd" (OuterVolumeSpecName: "kube-api-access-m7wpd") pod "881f31d8-d421-48d0-9978-9ea26f413b44" (UID: "881f31d8-d421-48d0-9978-9ea26f413b44"). InnerVolumeSpecName "kube-api-access-m7wpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:01.044210 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.044170    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk6bs\" (UniqueName: \"kubernetes.io/projected/8a92b351-087e-4c66-8660-c0df76ec3ba7-kube-api-access-rk6bs\") pod \"8a92b351-087e-4c66-8660-c0df76ec3ba7\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") "
Sep 10 21:35:01.044406 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.044237    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a92b351-087e-4c66-8660-c0df76ec3ba7-plugins-config-volume\") pod \"8a92b351-087e-4c66-8660-c0df76ec3ba7\" (UID: \"8a92b351-087e-4c66-8660-c0df76ec3ba7\") "
Sep 10 21:35:01.044595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.044577    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7wpd\" (UniqueName: \"kubernetes.io/projected/881f31d8-d421-48d0-9978-9ea26f413b44-kube-api-access-m7wpd\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:35:01.044674 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.044598    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/881f31d8-d421-48d0-9978-9ea26f413b44-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:35:01.044674 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.044656    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a92b351-087e-4c66-8660-c0df76ec3ba7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a92b351-087e-4c66-8660-c0df76ec3ba7" (UID: "8a92b351-087e-4c66-8660-c0df76ec3ba7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:01.046321 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.046303    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a92b351-087e-4c66-8660-c0df76ec3ba7-kube-api-access-rk6bs" (OuterVolumeSpecName: "kube-api-access-rk6bs") pod "8a92b351-087e-4c66-8660-c0df76ec3ba7" (UID: "8a92b351-087e-4c66-8660-c0df76ec3ba7"). InnerVolumeSpecName "kube-api-access-rk6bs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:01.138077 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.137992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:01.145416 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.145395    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk6bs\" (UniqueName: \"kubernetes.io/projected/8a92b351-087e-4c66-8660-c0df76ec3ba7-kube-api-access-rk6bs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:35:01.145534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.145419    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a92b351-087e-4c66-8660-c0df76ec3ba7-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:35:01.265381 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.265339    2583 generic.go:356] "Generic (PLEG): container finished" podID="881f31d8-d421-48d0-9978-9ea26f413b44" containerID="fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9" exitCode=0
Sep 10 21:35:01.265608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.265392    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"
Sep 10 21:35:01.265608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.265421    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz" event={"ID":"881f31d8-d421-48d0-9978-9ea26f413b44","Type":"ContainerDied","Data":"fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9"}
Sep 10 21:35:01.265608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.265478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz" event={"ID":"881f31d8-d421-48d0-9978-9ea26f413b44","Type":"ContainerDied","Data":"597adc54852fb8b7882c1d88673af492e77a6192b3802afa03718faf67f9bad0"}
Sep 10 21:35:01.265608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.265529    2583 scope.go:117] "RemoveContainer" containerID="fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9"
Sep 10 21:35:01.266838 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.266816    2583 generic.go:356] "Generic (PLEG): container finished" podID="8a92b351-087e-4c66-8660-c0df76ec3ba7" containerID="9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697" exitCode=0
Sep 10 21:35:01.266960 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.266860    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"
Sep 10 21:35:01.266960 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.266911    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6" event={"ID":"8a92b351-087e-4c66-8660-c0df76ec3ba7","Type":"ContainerDied","Data":"9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697"}
Sep 10 21:35:01.267075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.266962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6" event={"ID":"8a92b351-087e-4c66-8660-c0df76ec3ba7","Type":"ContainerDied","Data":"9bf2d8010754d853f6678f618c89bb6db34d0d7c5dfc59d1ac5ca8c0153b6952"}
Sep 10 21:35:01.275692 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.275675    2583 scope.go:117] "RemoveContainer" containerID="fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9"
Sep 10 21:35:01.275983 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:35:01.275962    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9\": container with ID starting with fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9 not found: ID does not exist" containerID="fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9"
Sep 10 21:35:01.276042 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.275991    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9"} err="failed to get container status \"fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9\": rpc error: code = NotFound desc = could not find container \"fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9\": container with ID starting with fab2140d97c2a830fbfb3d253ee3eccf3b9d34bbb13a66781331756156cc7aa9 not found: ID does not exist"
Sep 10 21:35:01.276042 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.276007    2583 scope.go:117] "RemoveContainer" containerID="9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697"
Sep 10 21:35:01.283967 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.283951    2583 scope.go:117] "RemoveContainer" containerID="9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697"
Sep 10 21:35:01.284226 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:35:01.284210    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697\": container with ID starting with 9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697 not found: ID does not exist" containerID="9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697"
Sep 10 21:35:01.284272 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.284232    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697"} err="failed to get container status \"9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697\": rpc error: code = NotFound desc = could not find container \"9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697\": container with ID starting with 9d382595ad6fa40a71b15a39eac14c876956763e7c04548a2f791b56cfb67697 not found: ID does not exist"
Sep 10 21:35:01.289787 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.289763    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"]
Sep 10 21:35:01.295293 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.295269    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bd7b8a-77db5fccb5-kcfrz"]
Sep 10 21:35:01.304765 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.304743    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"]
Sep 10 21:35:01.309390 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.309367    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bd7b8a-657fdc8b46-pvtd6"]
Sep 10 21:35:01.960434 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.960399    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f680f7e-db2d-4b11-b45d-1d2299ffc258" path="/var/lib/kubelet/pods/3f680f7e-db2d-4b11-b45d-1d2299ffc258/volumes"
Sep 10 21:35:01.960808 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.960775    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881f31d8-d421-48d0-9978-9ea26f413b44" path="/var/lib/kubelet/pods/881f31d8-d421-48d0-9978-9ea26f413b44/volumes"
Sep 10 21:35:01.961088 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:01.961075    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a92b351-087e-4c66-8660-c0df76ec3ba7" path="/var/lib/kubelet/pods/8a92b351-087e-4c66-8660-c0df76ec3ba7/volumes"
Sep 10 21:35:06.966034 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:06.966004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:07.035792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:07.035742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:07.228010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:07.227895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:07.813310 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:07.813260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:09.615979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:09.615943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:10.023094 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:10.023066    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:35:10.023799 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:10.023776    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:35:10.117064 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:10.117040    2583 scope.go:117] "RemoveContainer" containerID="acd0d68d9d96ed6de798b4eb18e104b81ca9bbd73666b1a787cbf22d3d3ac8ea"
Sep 10 21:35:10.756303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:10.756265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:11.156521 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.156481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:11.440244 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440159    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"]
Sep 10 21:35:11.440807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440788    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a92b351-087e-4c66-8660-c0df76ec3ba7" containerName="payload-pre-processing"
Sep 10 21:35:11.440863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440812    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a92b351-087e-4c66-8660-c0df76ec3ba7" containerName="payload-pre-processing"
Sep 10 21:35:11.440863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440831    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f680f7e-db2d-4b11-b45d-1d2299ffc258" containerName="revoke-keys"
Sep 10 21:35:11.440863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440840    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f680f7e-db2d-4b11-b45d-1d2299ffc258" containerName="revoke-keys"
Sep 10 21:35:11.440955 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440864    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881f31d8-d421-48d0-9978-9ea26f413b44" containerName="payload-processing"
Sep 10 21:35:11.440955 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440874    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="881f31d8-d421-48d0-9978-9ea26f413b44" containerName="payload-processing"
Sep 10 21:35:11.441021 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440974    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="881f31d8-d421-48d0-9978-9ea26f413b44" containerName="payload-processing"
Sep 10 21:35:11.441021 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.440990    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f680f7e-db2d-4b11-b45d-1d2299ffc258" containerName="revoke-keys"
Sep 10 21:35:11.441021 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.441000    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a92b351-087e-4c66-8660-c0df76ec3ba7" containerName="payload-pre-processing"
Sep 10 21:35:11.444481 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.444458    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.447601 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.447576    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-c8d758-openshift-default-dockercfg-kpscz\""
Sep 10 21:35:11.447803 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.447783    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-podinfo\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.447879 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.447823    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2587439-0cd9-4318-bb39-4a1c85b405fc-istiod-ca-cert\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.447936 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.447883    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-envoy\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.447998 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.447931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-socket\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.447998 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.447957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xzgx\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-kube-api-access-6xzgx\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.447998 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.447990    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-data\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.448162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.448039    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-certs\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.448162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.448101    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-credential-socket\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.448162 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.448146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-token\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.458483 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.458459    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"]
Sep 10 21:35:11.471567 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.471536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:11.549526 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.549465    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-podinfo\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.550283 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.550259    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2587439-0cd9-4318-bb39-4a1c85b405fc-istiod-ca-cert\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.551313 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.551211    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2587439-0cd9-4318-bb39-4a1c85b405fc-istiod-ca-cert\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.551443 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.551281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-envoy\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.551923 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.551904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-socket\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.552364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.552348    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xzgx\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-kube-api-access-6xzgx\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.552493 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.552477    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-data\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.552655 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.552640    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-certs\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.552788 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.552774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-credential-socket\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.552904 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.552889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-token\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.553230 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.553143    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-data\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.553338 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.552310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-socket\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.553518 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.553451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-credential-socket\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.553696 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.553627    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-certs\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.557658 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.557637    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-envoy\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.557771 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.557747    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-podinfo\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.560629 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.560607    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-token\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.560873 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.560853    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xzgx\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-kube-api-access-6xzgx\") pod \"e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") " pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:11.717813 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.717723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:11.754796 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:11.754764    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:12.037892 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.037867    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"]
Sep 10 21:35:12.040129 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:35:12.040099    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2587439_0cd9_4318_bb39_4a1c85b405fc.slice/crio-2dadbd26b614bdf735d3d973693fc17003aae8c3316567cab30d0bf4108cf242 WatchSource:0}: Error finding container 2dadbd26b614bdf735d3d973693fc17003aae8c3316567cab30d0bf4108cf242: Status 404 returned error can't find the container with id 2dadbd26b614bdf735d3d973693fc17003aae8c3316567cab30d0bf4108cf242
Sep 10 21:35:12.042618 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.042577    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:35:12.042703 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.042668    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:35:12.042802 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.042783    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:35:12.329055 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.328962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" event={"ID":"c2587439-0cd9-4318-bb39-4a1c85b405fc","Type":"ContainerStarted","Data":"5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180"}
Sep 10 21:35:12.329055 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.329020    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" event={"ID":"c2587439-0cd9-4318-bb39-4a1c85b405fc","Type":"ContainerStarted","Data":"2dadbd26b614bdf735d3d973693fc17003aae8c3316567cab30d0bf4108cf242"}
Sep 10 21:35:12.353113 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.353053    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podStartSLOduration=1.35303767 podStartE2EDuration="1.35303767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:12.352214519 +0000 UTC m=+1502.879178865" watchObservedRunningTime="2026-09-10 21:35:12.35303767 +0000 UTC m=+1502.880002016"
Sep 10 21:35:12.755521 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:12.755457    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:13.760624 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:13.760591    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:14.210132 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:14.210104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:14.310698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:14.310661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:14.340746 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:14.340717    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:14.341632 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:14.341610    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:15.400356 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:15.400325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:15.500891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:15.500855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:15.721004 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:15.720913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:16.114625 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.114482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:16.213148 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.213110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:16.228284 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.228256    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"]
Sep 10 21:35:16.231945 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.231930    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.233834 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.233813    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c8d758\""
Sep 10 21:35:16.233834 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.233829    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c8d758-dockercfg-bx5w9\""
Sep 10 21:35:16.243646 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.243623    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"]
Sep 10 21:35:16.299169 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.299135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgbxp\" (UniqueName: \"kubernetes.io/projected/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-kube-api-access-dgbxp\") pod \"payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.299346 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.299193    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.400768 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.400733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgbxp\" (UniqueName: \"kubernetes.io/projected/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-kube-api-access-dgbxp\") pod \"payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.401169 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.400792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.401568 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.401542    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.410200 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.410177    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgbxp\" (UniqueName: \"kubernetes.io/projected/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-kube-api-access-dgbxp\") pod \"payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.544307 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.544269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:16.842793 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:16.842769    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"]
Sep 10 21:35:16.844371 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:35:16.844339    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda879c2b5_1c3b_46ba_8f8d_38e00d3ffe8e.slice/crio-b15fb4f73ceaed34d38acdb8622265521fc3f91635411e8d40d29c600627aa95 WatchSource:0}: Error finding container b15fb4f73ceaed34d38acdb8622265521fc3f91635411e8d40d29c600627aa95: Status 404 returned error can't find the container with id b15fb4f73ceaed34d38acdb8622265521fc3f91635411e8d40d29c600627aa95
Sep 10 21:35:17.371210 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:17.371161    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm" event={"ID":"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e","Type":"ContainerStarted","Data":"e62aa07586491fab20a4317ce69fa8e60da14f0c112a4575784ada2bb5c9b932"}
Sep 10 21:35:17.371404 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:17.371234    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm" event={"ID":"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e","Type":"ContainerStarted","Data":"b15fb4f73ceaed34d38acdb8622265521fc3f91635411e8d40d29c600627aa95"}
Sep 10 21:35:17.372092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:17.372064    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:17.390235 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:17.390182    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm" podStartSLOduration=1.390166567 podStartE2EDuration="1.390166567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:17.388010853 +0000 UTC m=+1507.914975190" watchObservedRunningTime="2026-09-10 21:35:17.390166567 +0000 UTC m=+1507.917130913"
Sep 10 21:35:17.772710 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:17.772675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:18.421927 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.421898    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:18.427486 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.427457    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"]
Sep 10 21:35:18.431825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.431799    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5e6835-29817930-cbtjk"]
Sep 10 21:35:18.455006 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.454977    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"]
Sep 10 21:35:18.455375 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.455338    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf" podUID="755f26c0-c5dd-435f-9ee8-e945f3caebf1" containerName="payload-processing" containerID="cri-o://3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee" gracePeriod=30
Sep 10 21:35:18.525675 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.525640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:18.626122 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.625974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:18.692337 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.692252    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:18.718527 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.718477    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:35:18.826697 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.826669    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/755f26c0-c5dd-435f-9ee8-e945f3caebf1-plugins-config-volume\") pod \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") "
Sep 10 21:35:18.827093 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.826711    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqw8t\" (UniqueName: \"kubernetes.io/projected/755f26c0-c5dd-435f-9ee8-e945f3caebf1-kube-api-access-pqw8t\") pod \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\" (UID: \"755f26c0-c5dd-435f-9ee8-e945f3caebf1\") "
Sep 10 21:35:18.827093 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.827048    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755f26c0-c5dd-435f-9ee8-e945f3caebf1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "755f26c0-c5dd-435f-9ee8-e945f3caebf1" (UID: "755f26c0-c5dd-435f-9ee8-e945f3caebf1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:18.828855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.828828    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755f26c0-c5dd-435f-9ee8-e945f3caebf1-kube-api-access-pqw8t" (OuterVolumeSpecName: "kube-api-access-pqw8t") pod "755f26c0-c5dd-435f-9ee8-e945f3caebf1" (UID: "755f26c0-c5dd-435f-9ee8-e945f3caebf1"). InnerVolumeSpecName "kube-api-access-pqw8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:18.927599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.927566    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/755f26c0-c5dd-435f-9ee8-e945f3caebf1-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:35:18.927599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:18.927593    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqw8t\" (UniqueName: \"kubernetes.io/projected/755f26c0-c5dd-435f-9ee8-e945f3caebf1-kube-api-access-pqw8t\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:35:19.013764 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.013672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:19.385010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.384923    2583 generic.go:356] "Generic (PLEG): container finished" podID="755f26c0-c5dd-435f-9ee8-e945f3caebf1" containerID="3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee" exitCode=0
Sep 10 21:35:19.385010 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.384965    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"
Sep 10 21:35:19.385250 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.385013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf" event={"ID":"755f26c0-c5dd-435f-9ee8-e945f3caebf1","Type":"ContainerDied","Data":"3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee"}
Sep 10 21:35:19.385250 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.385069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf" event={"ID":"755f26c0-c5dd-435f-9ee8-e945f3caebf1","Type":"ContainerDied","Data":"31caabad7ad3c9866ae585658e3aa63dda1a8048fb21c1b3647858aee0cf64e8"}
Sep 10 21:35:19.385250 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.385102    2583 scope.go:117] "RemoveContainer" containerID="3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee"
Sep 10 21:35:19.395322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.395305    2583 scope.go:117] "RemoveContainer" containerID="3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee"
Sep 10 21:35:19.395632 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:35:19.395611    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee\": container with ID starting with 3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee not found: ID does not exist" containerID="3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee"
Sep 10 21:35:19.395706 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.395641    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee"} err="failed to get container status \"3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee\": rpc error: code = NotFound desc = could not find container \"3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee\": container with ID starting with 3b5495ebb47eb2db017f38152a00e5581a97ac2350b2f4a582a6e04e2db031ee not found: ID does not exist"
Sep 10 21:35:19.408461 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.408434    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"]
Sep 10 21:35:19.415883 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.415866    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5e6835-6bf784c587-fmktf"]
Sep 10 21:35:19.961081 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.961042    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="755f26c0-c5dd-435f-9ee8-e945f3caebf1" path="/var/lib/kubelet/pods/755f26c0-c5dd-435f-9ee8-e945f3caebf1/volumes"
Sep 10 21:35:19.963286 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:19.963255    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b664ed25-3ddf-4f5f-a253-c5191b47e870" path="/var/lib/kubelet/pods/b664ed25-3ddf-4f5f-a253-c5191b47e870/volumes"
Sep 10 21:35:21.389474 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:21.389437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:21.491685 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:21.491646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:25.165454 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:25.165409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:25.239724 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:25.239688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:25.434126 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:25.434013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:29.387474 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:29.387440    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:35:30.382675 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:30.382636    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:35:30.382885 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:30.382692    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:35:31.466195 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:31.466165    2583 generic.go:356] "Generic (PLEG): container finished" podID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerID="5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180" exitCode=0
Sep 10 21:35:31.466649 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:31.466245    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" event={"ID":"c2587439-0cd9-4318-bb39-4a1c85b405fc","Type":"ContainerDied","Data":"5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180"}
Sep 10 21:35:31.466699 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:31.466651    2583 scope.go:117] "RemoveContainer" containerID="5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180"
Sep 10 21:35:31.467757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:31.467719    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:35:31.467863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:31.467796    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:35:31.467863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:31.467831    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 21:35:32.472946 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:32.472872    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" event={"ID":"c2587439-0cd9-4318-bb39-4a1c85b405fc","Type":"ContainerStarted","Data":"a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a"}
Sep 10 21:35:32.755910 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:32.755825    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:32.756063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:32.756044    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 10 21:35:32.756106 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:32.756089    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 10 21:35:33.755812 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:33.755779    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 10 21:35:33.756311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:33.755853    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 10 21:35:34.755694 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:34.755659    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 10 21:35:34.755876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:34.755724    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 10 21:35:35.755258 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:35.755217    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 10 21:35:35.755452 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:35.755291    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 10 21:35:36.720064 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:36.720024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:36.755837 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:36.755802    2583 patch_prober.go:28] interesting pod/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 10 21:35:36.756003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:36.755855    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 10 21:35:37.759070 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:37.759031    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:37.759457 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:37.759289    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:37.760019 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:37.759999    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:35:39.191451 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:39.191415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:39.866529 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:39.866446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:40.226265 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:40.226229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:40.388448 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:40.388415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:43.248980 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:43.248929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:43.339738 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:43.339702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:44.410120 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:44.410083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:44.510346 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:44.510301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:44.591932 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:44.591899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:45.020647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.020616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:45.120181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.120146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:45.210423 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.210390    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"]
Sep 10 21:35:45.210824 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.210811    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755f26c0-c5dd-435f-9ee8-e945f3caebf1" containerName="payload-processing"
Sep 10 21:35:45.210873 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.210826    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="755f26c0-c5dd-435f-9ee8-e945f3caebf1" containerName="payload-processing"
Sep 10 21:35:45.210908 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.210899    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="755f26c0-c5dd-435f-9ee8-e945f3caebf1" containerName="payload-processing"
Sep 10 21:35:45.213327 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.213309    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.218496 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.218466    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5f25ef-dockercfg-rjcxg\""
Sep 10 21:35:45.218833 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.218814    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5f25ef\""
Sep 10 21:35:45.229676 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.229650    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"]
Sep 10 21:35:45.309045 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.308968    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"]
Sep 10 21:35:45.311828 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.311811    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.318598 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.318571    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"]
Sep 10 21:35:45.395647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.395611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b75b35-a805-444e-b3b1-e02b161fc2eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.395826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.395659    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7rts\" (UniqueName: \"kubernetes.io/projected/54b75b35-a805-444e-b3b1-e02b161fc2eb-kube-api-access-j7rts\") pod \"payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.497007 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.496970    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.497374 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.497022    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b75b35-a805-444e-b3b1-e02b161fc2eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.497374 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.497050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7rts\" (UniqueName: \"kubernetes.io/projected/54b75b35-a805-444e-b3b1-e02b161fc2eb-kube-api-access-j7rts\") pod \"payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.497374 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.497102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nrst\" (UniqueName: \"kubernetes.io/projected/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-kube-api-access-8nrst\") pod \"payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.497888 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.497854    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b75b35-a805-444e-b3b1-e02b161fc2eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.507390 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.507358    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7rts\" (UniqueName: \"kubernetes.io/projected/54b75b35-a805-444e-b3b1-e02b161fc2eb-kube-api-access-j7rts\") pod \"payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.531102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.531068    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:45.598360 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.598324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nrst\" (UniqueName: \"kubernetes.io/projected/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-kube-api-access-8nrst\") pod \"payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.598494 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.598420    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.599442 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.599419    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.613668 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.613611    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nrst\" (UniqueName: \"kubernetes.io/projected/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-kube-api-access-8nrst\") pod \"payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.623295 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.623270    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:45.804280 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.804252    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"]
Sep 10 21:35:45.805753 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:35:45.805728    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b75b35_a805_444e_b3b1_e02b161fc2eb.slice/crio-cb1f3b7b3b0639aa858b7c168d7b2fb5082535b913e98bb6c0c3a35af8074eaa WatchSource:0}: Error finding container cb1f3b7b3b0639aa858b7c168d7b2fb5082535b913e98bb6c0c3a35af8074eaa: Status 404 returned error can't find the container with id cb1f3b7b3b0639aa858b7c168d7b2fb5082535b913e98bb6c0c3a35af8074eaa
Sep 10 21:35:45.898243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:45.898068    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"]
Sep 10 21:35:45.900527 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:35:45.900471    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aca76a4_8c91_4bd2_ad3d_cd9a4fb97364.slice/crio-c5932927f5c4af030ec3cb1d1df0dcb9e663bc514521e4270406ad6f63817bf4 WatchSource:0}: Error finding container c5932927f5c4af030ec3cb1d1df0dcb9e663bc514521e4270406ad6f63817bf4: Status 404 returned error can't find the container with id c5932927f5c4af030ec3cb1d1df0dcb9e663bc514521e4270406ad6f63817bf4
Sep 10 21:35:46.558020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.557982    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7" event={"ID":"54b75b35-a805-444e-b3b1-e02b161fc2eb","Type":"ContainerStarted","Data":"6a379af6d69110385c5221f1c36aec987446bc9929a3b05be34406442c0e5f3c"}
Sep 10 21:35:46.558539 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.558042    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7" event={"ID":"54b75b35-a805-444e-b3b1-e02b161fc2eb","Type":"ContainerStarted","Data":"cb1f3b7b3b0639aa858b7c168d7b2fb5082535b913e98bb6c0c3a35af8074eaa"}
Sep 10 21:35:46.558539 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.558070    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:35:46.559708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.559683    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt" event={"ID":"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364","Type":"ContainerStarted","Data":"3d12eae0d3bacf4950a6af87594c7786f507a2c58ca5cff25e4ff42fa9f07ba2"}
Sep 10 21:35:46.559820 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.559730    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt" event={"ID":"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364","Type":"ContainerStarted","Data":"c5932927f5c4af030ec3cb1d1df0dcb9e663bc514521e4270406ad6f63817bf4"}
Sep 10 21:35:46.559820 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.559799    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:46.575125 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.575078    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7" podStartSLOduration=1.575049292 podStartE2EDuration="1.575049292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:46.57269682 +0000 UTC m=+1537.099661168" watchObservedRunningTime="2026-09-10 21:35:46.575049292 +0000 UTC m=+1537.102013647"
Sep 10 21:35:46.588287 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:46.588232    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt" podStartSLOduration=1.588214841 podStartE2EDuration="1.588214841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:46.586928421 +0000 UTC m=+1537.113892804" watchObservedRunningTime="2026-09-10 21:35:46.588214841 +0000 UTC m=+1537.115179187"
Sep 10 21:35:56.687584 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:56.687542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:56.748968 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:56.748929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:57.567282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:57.567254    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:35:57.567454 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:35:57.567320    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:36:09.173694 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:09.173655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:09.274001 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:09.273962    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:10.153557 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:10.153526    2583 scope.go:117] "RemoveContainer" containerID="4f9947f166fdd506a28d3fde5ccdf3c61e54f0df718ef533dfa6907133483f08"
Sep 10 21:36:13.209635 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:13.209600    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qzrg6_c273c2b1-fd37-4dcf-a888-17d3a8438bf9/payload-processing/0.log"
Sep 10 21:36:13.324369 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:13.324341    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt_2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364/payload-processing/0.log"
Sep 10 21:36:17.040879 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:17.040839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:40.863305 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:40.863276    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qzrg6_c273c2b1-fd37-4dcf-a888-17d3a8438bf9/payload-processing/0.log"
Sep 10 21:36:40.980947 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:40.980920    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt_2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364/payload-processing/0.log"
Sep 10 21:36:41.258275 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:41.258237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:41.633182 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:41.633089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:41.987900 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:41.987867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:44.565347 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:44.565311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:44.663441 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:44.663412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:45.707941 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:45.707903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:45.803901 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:45.803868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:46.202594 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.202562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:46.294049 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.294001    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:46.319097 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.319064    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"]
Sep 10 21:36:46.322328 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.322307    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.324282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.324249    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-389568\""
Sep 10 21:36:46.324401 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.324384    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-389568-dockercfg-kccz2\""
Sep 10 21:36:46.331888 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.331863    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"]
Sep 10 21:36:46.368319 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.368283    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6aa50f-959e-4d54-8bd2-331100307ca1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.368488 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.368341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wg9n\" (UniqueName: \"kubernetes.io/projected/da6aa50f-959e-4d54-8bd2-331100307ca1-kube-api-access-2wg9n\") pod \"payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.418528 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.400632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:46.469658 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.469574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wg9n\" (UniqueName: \"kubernetes.io/projected/da6aa50f-959e-4d54-8bd2-331100307ca1-kube-api-access-2wg9n\") pod \"payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.469809 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.469728    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6aa50f-959e-4d54-8bd2-331100307ca1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.470709 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.470687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6aa50f-959e-4d54-8bd2-331100307ca1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.476615 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.476592    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wg9n\" (UniqueName: \"kubernetes.io/projected/da6aa50f-959e-4d54-8bd2-331100307ca1-kube-api-access-2wg9n\") pod \"payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.635198 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.635159    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:46.912715 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:46.912689    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"]
Sep 10 21:36:46.914961 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:36:46.914930    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6aa50f_959e_4d54_8bd2_331100307ca1.slice/crio-64bd7afddfc87f9ba28caf5ff9ec91b83298349bbe00e0b1c4ad9a78101bb15b WatchSource:0}: Error finding container 64bd7afddfc87f9ba28caf5ff9ec91b83298349bbe00e0b1c4ad9a78101bb15b: Status 404 returned error can't find the container with id 64bd7afddfc87f9ba28caf5ff9ec91b83298349bbe00e0b1c4ad9a78101bb15b
Sep 10 21:36:47.891089 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:47.891053    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4" event={"ID":"da6aa50f-959e-4d54-8bd2-331100307ca1","Type":"ContainerStarted","Data":"08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b"}
Sep 10 21:36:47.891266 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:47.891103    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4" event={"ID":"da6aa50f-959e-4d54-8bd2-331100307ca1","Type":"ContainerStarted","Data":"64bd7afddfc87f9ba28caf5ff9ec91b83298349bbe00e0b1c4ad9a78101bb15b"}
Sep 10 21:36:47.891266 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:47.891171    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:36:47.907488 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:47.907437    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4" podStartSLOduration=1.907420667 podStartE2EDuration="1.907420667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:47.905674594 +0000 UTC m=+1598.432638942" watchObservedRunningTime="2026-09-10 21:36:47.907420667 +0000 UTC m=+1598.434385026"
Sep 10 21:36:58.900231 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:36:58.900195    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:37:09.482234 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:09.482199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:10.169141 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.169109    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"]
Sep 10 21:37:10.172105 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.172089    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"
Sep 10 21:37:10.173891 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.173870    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:37:10.177818 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.177796    2583 scope.go:117] "RemoveContainer" containerID="0e6f9ff43b23145719c356e767c33d7318094506fb30a32f8473728499ac7539"
Sep 10 21:37:10.179930 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.179908    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"]
Sep 10 21:37:10.307186 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.307138    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ca5da-b92d-4fc6-b447-251a95600876-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx\" (UID: \"2d3ca5da-b92d-4fc6-b447-251a95600876\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"
Sep 10 21:37:10.407864 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.407830    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ca5da-b92d-4fc6-b447-251a95600876-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx\" (UID: \"2d3ca5da-b92d-4fc6-b447-251a95600876\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"
Sep 10 21:37:10.408637 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.408614    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ca5da-b92d-4fc6-b447-251a95600876-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx\" (UID: \"2d3ca5da-b92d-4fc6-b447-251a95600876\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"
Sep 10 21:37:10.485907 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.485813    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"
Sep 10 21:37:10.759826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:10.759802    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"]
Sep 10 21:37:10.761889 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:37:10.761856    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3ca5da_b92d_4fc6_b447_251a95600876.slice/crio-c04668fbbe8f82b3718a3b1a7a85d277d4f12caf798c9d267065c24bf9cd06b0 WatchSource:0}: Error finding container c04668fbbe8f82b3718a3b1a7a85d277d4f12caf798c9d267065c24bf9cd06b0: Status 404 returned error can't find the container with id c04668fbbe8f82b3718a3b1a7a85d277d4f12caf798c9d267065c24bf9cd06b0
Sep 10 21:37:11.025310 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:11.025214    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx" event={"ID":"2d3ca5da-b92d-4fc6-b447-251a95600876","Type":"ContainerStarted","Data":"ba9ca012b910a37eb42659975a56851ffa53ed44ecaa2f320cf05111c043a3a0"}
Sep 10 21:37:11.025310 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:11.025265    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx" event={"ID":"2d3ca5da-b92d-4fc6-b447-251a95600876","Type":"ContainerStarted","Data":"c04668fbbe8f82b3718a3b1a7a85d277d4f12caf798c9d267065c24bf9cd06b0"}
Sep 10 21:37:11.040318 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:11.040270    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx" podStartSLOduration=1.040254942 podStartE2EDuration="1.040254942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:37:11.037819372 +0000 UTC m=+1621.564783720" watchObservedRunningTime="2026-09-10 21:37:11.040254942 +0000 UTC m=+1621.567219289"
Sep 10 21:37:12.031871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:12.031836    2583 generic.go:356] "Generic (PLEG): container finished" podID="2d3ca5da-b92d-4fc6-b447-251a95600876" containerID="ba9ca012b910a37eb42659975a56851ffa53ed44ecaa2f320cf05111c043a3a0" exitCode=0
Sep 10 21:37:12.032338 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:12.031916    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx" event={"ID":"2d3ca5da-b92d-4fc6-b447-251a95600876","Type":"ContainerDied","Data":"ba9ca012b910a37eb42659975a56851ffa53ed44ecaa2f320cf05111c043a3a0"}
Sep 10 21:37:13.180942 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:13.180917    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"
Sep 10 21:37:13.336149 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:13.336052    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ca5da-b92d-4fc6-b447-251a95600876-maas-api-service-ca\") pod \"2d3ca5da-b92d-4fc6-b447-251a95600876\" (UID: \"2d3ca5da-b92d-4fc6-b447-251a95600876\") "
Sep 10 21:37:13.336464 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:13.336436    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3ca5da-b92d-4fc6-b447-251a95600876-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d3ca5da-b92d-4fc6-b447-251a95600876" (UID: "2d3ca5da-b92d-4fc6-b447-251a95600876"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:13.437262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:13.437224    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ca5da-b92d-4fc6-b447-251a95600876-maas-api-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.047245 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:14.047212    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"
Sep 10 21:37:14.047408 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:14.047242    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx" event={"ID":"2d3ca5da-b92d-4fc6-b447-251a95600876","Type":"ContainerDied","Data":"c04668fbbe8f82b3718a3b1a7a85d277d4f12caf798c9d267065c24bf9cd06b0"}
Sep 10 21:37:14.047408 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:14.047287    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c04668fbbe8f82b3718a3b1a7a85d277d4f12caf798c9d267065c24bf9cd06b0"
Sep 10 21:37:16.254734 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.254701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:16.281257 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.281225    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"]
Sep 10 21:37:16.281470 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.281435    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4" podUID="da6aa50f-959e-4d54-8bd2-331100307ca1" containerName="payload-processing" containerID="cri-o://08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b" gracePeriod=30
Sep 10 21:37:16.355181 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.355107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:16.455335 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.455300    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"]
Sep 10 21:37:16.458573 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.457960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:16.462620 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.462597    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389568-vd7vx"]
Sep 10 21:37:16.538647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.538614    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:37:16.560317 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.560285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:16.668574 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.668538    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6aa50f-959e-4d54-8bd2-331100307ca1-plugins-config-volume\") pod \"da6aa50f-959e-4d54-8bd2-331100307ca1\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") "
Sep 10 21:37:16.668574 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.668581    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wg9n\" (UniqueName: \"kubernetes.io/projected/da6aa50f-959e-4d54-8bd2-331100307ca1-kube-api-access-2wg9n\") pod \"da6aa50f-959e-4d54-8bd2-331100307ca1\" (UID: \"da6aa50f-959e-4d54-8bd2-331100307ca1\") "
Sep 10 21:37:16.669589 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.669002    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da6aa50f-959e-4d54-8bd2-331100307ca1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da6aa50f-959e-4d54-8bd2-331100307ca1" (UID: "da6aa50f-959e-4d54-8bd2-331100307ca1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:16.671928 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.671900    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da6aa50f-959e-4d54-8bd2-331100307ca1-kube-api-access-2wg9n" (OuterVolumeSpecName: "kube-api-access-2wg9n") pod "da6aa50f-959e-4d54-8bd2-331100307ca1" (UID: "da6aa50f-959e-4d54-8bd2-331100307ca1"). InnerVolumeSpecName "kube-api-access-2wg9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:16.770051 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.769967    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6aa50f-959e-4d54-8bd2-331100307ca1-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.770051 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.769997    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wg9n\" (UniqueName: \"kubernetes.io/projected/da6aa50f-959e-4d54-8bd2-331100307ca1-kube-api-access-2wg9n\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:37:16.865025 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:16.864986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:17.065222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.065126    2583 generic.go:356] "Generic (PLEG): container finished" podID="da6aa50f-959e-4d54-8bd2-331100307ca1" containerID="08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b" exitCode=0
Sep 10 21:37:17.065222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.065173    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"
Sep 10 21:37:17.065463 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.065220    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4" event={"ID":"da6aa50f-959e-4d54-8bd2-331100307ca1","Type":"ContainerDied","Data":"08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b"}
Sep 10 21:37:17.065463 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.065284    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4" event={"ID":"da6aa50f-959e-4d54-8bd2-331100307ca1","Type":"ContainerDied","Data":"64bd7afddfc87f9ba28caf5ff9ec91b83298349bbe00e0b1c4ad9a78101bb15b"}
Sep 10 21:37:17.065463 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.065318    2583 scope.go:117] "RemoveContainer" containerID="08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b"
Sep 10 21:37:17.074393 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.074377    2583 scope.go:117] "RemoveContainer" containerID="08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b"
Sep 10 21:37:17.074692 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:37:17.074674    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b\": container with ID starting with 08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b not found: ID does not exist" containerID="08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b"
Sep 10 21:37:17.074771 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.074704    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b"} err="failed to get container status \"08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b\": rpc error: code = NotFound desc = could not find container \"08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b\": container with ID starting with 08aa4c4b3c6de4780f5e5972ca1ea5cc16e1d50f7085265a2470d2bbb87ce22b not found: ID does not exist"
Sep 10 21:37:17.085773 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.085738    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"]
Sep 10 21:37:17.092194 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.092158    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-389568-676794855c-wx5v4"]
Sep 10 21:37:17.961102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.961067    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3ca5da-b92d-4fc6-b447-251a95600876" path="/var/lib/kubelet/pods/2d3ca5da-b92d-4fc6-b447-251a95600876/volumes"
Sep 10 21:37:17.961488 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:17.961430    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da6aa50f-959e-4d54-8bd2-331100307ca1" path="/var/lib/kubelet/pods/da6aa50f-959e-4d54-8bd2-331100307ca1/volumes"
Sep 10 21:37:22.572192 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:22.572134    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.631747 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:22.631710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.821928 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:22.821895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:26.315269 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.315230    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"]
Sep 10 21:37:26.315770 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.315753    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d3ca5da-b92d-4fc6-b447-251a95600876" containerName="revoke-keys"
Sep 10 21:37:26.315816 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.315772    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3ca5da-b92d-4fc6-b447-251a95600876" containerName="revoke-keys"
Sep 10 21:37:26.315816 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.315798    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6aa50f-959e-4d54-8bd2-331100307ca1" containerName="payload-processing"
Sep 10 21:37:26.315816 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.315804    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6aa50f-959e-4d54-8bd2-331100307ca1" containerName="payload-processing"
Sep 10 21:37:26.315918 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.315868    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6aa50f-959e-4d54-8bd2-331100307ca1" containerName="payload-processing"
Sep 10 21:37:26.315918 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.315879    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d3ca5da-b92d-4fc6-b447-251a95600876" containerName="revoke-keys"
Sep 10 21:37:26.320222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.320204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"
Sep 10 21:37:26.322202 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.322176    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:37:26.324530 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.324479    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"]
Sep 10 21:37:26.362009 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.361977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35ef1f03-29cf-43ac-985a-c6b4b2dd1bca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc\" (UID: \"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"
Sep 10 21:37:26.463178 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.463141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35ef1f03-29cf-43ac-985a-c6b4b2dd1bca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc\" (UID: \"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"
Sep 10 21:37:26.463763 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.463745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35ef1f03-29cf-43ac-985a-c6b4b2dd1bca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc\" (UID: \"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"
Sep 10 21:37:26.521899 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.521867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:26.634185 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.634096    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"
Sep 10 21:37:26.903702 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:26.903677    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"]
Sep 10 21:37:26.905436 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:37:26.905411    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ef1f03_29cf_43ac_985a_c6b4b2dd1bca.slice/crio-9f3952273290f913bef45fd98029c3dccf3d817db12b4c0009c8fa2779935391 WatchSource:0}: Error finding container 9f3952273290f913bef45fd98029c3dccf3d817db12b4c0009c8fa2779935391: Status 404 returned error can't find the container with id 9f3952273290f913bef45fd98029c3dccf3d817db12b4c0009c8fa2779935391
Sep 10 21:37:27.128863 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:27.128833    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc" event={"ID":"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca","Type":"ContainerStarted","Data":"9f3952273290f913bef45fd98029c3dccf3d817db12b4c0009c8fa2779935391"}
Sep 10 21:37:28.135073 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:28.135036    2583 generic.go:356] "Generic (PLEG): container finished" podID="35ef1f03-29cf-43ac-985a-c6b4b2dd1bca" containerID="4f16dc8b472e022b4cf9e558b1558d1c3346cd1add297ac02c4d3ac5e35a61c3" exitCode=0
Sep 10 21:37:28.135446 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:28.135114    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc" event={"ID":"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca","Type":"ContainerDied","Data":"4f16dc8b472e022b4cf9e558b1558d1c3346cd1add297ac02c4d3ac5e35a61c3"}
Sep 10 21:37:29.274675 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:29.274651    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"
Sep 10 21:37:29.389035 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:29.389002    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35ef1f03-29cf-43ac-985a-c6b4b2dd1bca-maas-api-service-ca\") pod \"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca\" (UID: \"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca\") "
Sep 10 21:37:29.389333 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:29.389310    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35ef1f03-29cf-43ac-985a-c6b4b2dd1bca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "35ef1f03-29cf-43ac-985a-c6b4b2dd1bca" (UID: "35ef1f03-29cf-43ac-985a-c6b4b2dd1bca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:29.490273 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:29.490190    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/35ef1f03-29cf-43ac-985a-c6b4b2dd1bca-maas-api-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:37:30.150183 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:30.150147    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc" event={"ID":"35ef1f03-29cf-43ac-985a-c6b4b2dd1bca","Type":"ContainerDied","Data":"9f3952273290f913bef45fd98029c3dccf3d817db12b4c0009c8fa2779935391"}
Sep 10 21:37:30.150352 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:30.150189    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"
Sep 10 21:37:30.150352 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:30.150196    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f3952273290f913bef45fd98029c3dccf3d817db12b4c0009c8fa2779935391"
Sep 10 21:37:32.140557 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:32.140496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:32.202220 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:32.202190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:41.968044 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:41.968003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:42.008065 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.008028    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"]
Sep 10 21:37:42.008612 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.008578    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm" podUID="a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" containerName="payload-pre-processing" containerID="cri-o://e62aa07586491fab20a4317ce69fa8e60da14f0c112a4575784ada2bb5c9b932" gracePeriod=30
Sep 10 21:37:42.031937 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.031900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:42.093613 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.093580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:42.172620 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.172578    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"]
Sep 10 21:37:42.175673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.175585    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:42.175913 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.175891    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-c8d758-bgcsc"]
Sep 10 21:37:42.218686 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.218594    2583 generic.go:356] "Generic (PLEG): container finished" podID="a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" containerID="e62aa07586491fab20a4317ce69fa8e60da14f0c112a4575784ada2bb5c9b932" exitCode=0
Sep 10 21:37:42.218826 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.218676    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm" event={"ID":"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e","Type":"ContainerDied","Data":"e62aa07586491fab20a4317ce69fa8e60da14f0c112a4575784ada2bb5c9b932"}
Sep 10 21:37:42.237231 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.237201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:42.264515 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.264477    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:37:42.418832 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.418796    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgbxp\" (UniqueName: \"kubernetes.io/projected/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-kube-api-access-dgbxp\") pod \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") "
Sep 10 21:37:42.419041 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.418875    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-plugins-config-volume\") pod \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\" (UID: \"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e\") "
Sep 10 21:37:42.419268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.419241    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" (UID: "a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:42.421038 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.421002    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-kube-api-access-dgbxp" (OuterVolumeSpecName: "kube-api-access-dgbxp") pod "a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" (UID: "a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e"). InnerVolumeSpecName "kube-api-access-dgbxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:42.520467 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.520373    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.520467 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.520409    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgbxp\" (UniqueName: \"kubernetes.io/projected/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e-kube-api-access-dgbxp\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:37:42.532222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:42.532189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:43.225116 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:43.225080    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm" event={"ID":"a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e","Type":"ContainerDied","Data":"b15fb4f73ceaed34d38acdb8622265521fc3f91635411e8d40d29c600627aa95"}
Sep 10 21:37:43.225673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:43.225134    2583 scope.go:117] "RemoveContainer" containerID="e62aa07586491fab20a4317ce69fa8e60da14f0c112a4575784ada2bb5c9b932"
Sep 10 21:37:43.225673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:43.225094    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"
Sep 10 21:37:43.245922 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:43.245899    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"]
Sep 10 21:37:43.251588 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:43.251568    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c8d758-7898866586-p76mm"]
Sep 10 21:37:43.960409 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:43.960377    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35ef1f03-29cf-43ac-985a-c6b4b2dd1bca" path="/var/lib/kubelet/pods/35ef1f03-29cf-43ac-985a-c6b4b2dd1bca/volumes"
Sep 10 21:37:43.960774 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:43.960761    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" path="/var/lib/kubelet/pods/a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e/volumes"
Sep 10 21:37:47.568536 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:37:47.568478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:10.203983 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:10.203947    2583 scope.go:117] "RemoveContainer" containerID="4fc5b6ce9fabafebc4e7a3ac7cf0283c2a66b34dad78613240e87edd8897f0d2"
Sep 10 21:38:24.431310 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:24.431265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:24.570561 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:24.570496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:30.150727 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.150674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:30.217065 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.217031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:30.396686 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.396649    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"]
Sep 10 21:38:30.396930 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.396905    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy" containerID="cri-o://a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a" gracePeriod=30
Sep 10 21:38:30.418281 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.418196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:30.773273 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.773193    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"]
Sep 10 21:38:30.773618 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.773605    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35ef1f03-29cf-43ac-985a-c6b4b2dd1bca" containerName="revoke-keys"
Sep 10 21:38:30.773671 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.773619    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="35ef1f03-29cf-43ac-985a-c6b4b2dd1bca" containerName="revoke-keys"
Sep 10 21:38:30.773671 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.773631    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" containerName="payload-pre-processing"
Sep 10 21:38:30.773671 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.773636    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" containerName="payload-pre-processing"
Sep 10 21:38:30.773770 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.773705    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a879c2b5-1c3b-46ba-8f8d-38e00d3ffe8e" containerName="payload-pre-processing"
Sep 10 21:38:30.773770 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.773713    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="35ef1f03-29cf-43ac-985a-c6b4b2dd1bca" containerName="revoke-keys"
Sep 10 21:38:30.776825 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.776809    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"
Sep 10 21:38:30.778589 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.778568    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wfptt\""
Sep 10 21:38:30.784264 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.784237    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"]
Sep 10 21:38:30.804208 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.804167    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7de44e8e-3431-4134-98d0-1a782acad777-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn\" (UID: \"7de44e8e-3431-4134-98d0-1a782acad777\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"
Sep 10 21:38:30.892492 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.892461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:30.904775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.904739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7de44e8e-3431-4134-98d0-1a782acad777-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn\" (UID: \"7de44e8e-3431-4134-98d0-1a782acad777\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"
Sep 10 21:38:30.905337 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:30.905317    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7de44e8e-3431-4134-98d0-1a782acad777-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn\" (UID: \"7de44e8e-3431-4134-98d0-1a782acad777\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"
Sep 10 21:38:31.090281 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:31.090195    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"
Sep 10 21:38:31.360933 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:31.360880    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"]
Sep 10 21:38:31.363606 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:38:31.363570    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de44e8e_3431_4134_98d0_1a782acad777.slice/crio-c4553d6ae41f2345f2420bfec477ff9d0ed88c84aaac911b234bc568493c08d7 WatchSource:0}: Error finding container c4553d6ae41f2345f2420bfec477ff9d0ed88c84aaac911b234bc568493c08d7: Status 404 returned error can't find the container with id c4553d6ae41f2345f2420bfec477ff9d0ed88c84aaac911b234bc568493c08d7
Sep 10 21:38:31.474477 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:31.474439    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn" event={"ID":"7de44e8e-3431-4134-98d0-1a782acad777","Type":"ContainerStarted","Data":"c4553d6ae41f2345f2420bfec477ff9d0ed88c84aaac911b234bc568493c08d7"}
Sep 10 21:38:32.481865 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:32.481833    2583 generic.go:356] "Generic (PLEG): container finished" podID="7de44e8e-3431-4134-98d0-1a782acad777" containerID="a94d6fb77b8c12eda9e3ba7b1b2da2ab4bab0a2dbb92671fd0ae9c51227618cc" exitCode=0
Sep 10 21:38:32.482268 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:32.481886    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn" event={"ID":"7de44e8e-3431-4134-98d0-1a782acad777","Type":"ContainerDied","Data":"a94d6fb77b8c12eda9e3ba7b1b2da2ab4bab0a2dbb92671fd0ae9c51227618cc"}
Sep 10 21:38:33.617665 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:33.617641    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"
Sep 10 21:38:33.632227 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:33.632193    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7de44e8e-3431-4134-98d0-1a782acad777-maas-api-service-ca\") pod \"7de44e8e-3431-4134-98d0-1a782acad777\" (UID: \"7de44e8e-3431-4134-98d0-1a782acad777\") "
Sep 10 21:38:33.632706 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:33.632667    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de44e8e-3431-4134-98d0-1a782acad777-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7de44e8e-3431-4134-98d0-1a782acad777" (UID: "7de44e8e-3431-4134-98d0-1a782acad777"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:33.733938 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:33.733902    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7de44e8e-3431-4134-98d0-1a782acad777-maas-api-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:34.493288 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:34.493259    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"
Sep 10 21:38:34.493439 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:34.493298    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn" event={"ID":"7de44e8e-3431-4134-98d0-1a782acad777","Type":"ContainerDied","Data":"c4553d6ae41f2345f2420bfec477ff9d0ed88c84aaac911b234bc568493c08d7"}
Sep 10 21:38:34.493439 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:34.493348    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4553d6ae41f2345f2420bfec477ff9d0ed88c84aaac911b234bc568493c08d7"
Sep 10 21:38:35.651748 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.651721    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:38:35.753189 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753098    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-socket\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753189 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753138    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-token\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753189 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753162    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xzgx\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-kube-api-access-6xzgx\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753189 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753187    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-certs\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753236    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-data\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753277    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-credential-socket\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753317    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2587439-0cd9-4318-bb39-4a1c85b405fc-istiod-ca-cert\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753343    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-podinfo\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753549 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753408    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-envoy\") pod \"c2587439-0cd9-4318-bb39-4a1c85b405fc\" (UID: \"c2587439-0cd9-4318-bb39-4a1c85b405fc\") "
Sep 10 21:38:35.753801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753408    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:35.753801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753641    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:35.753801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753623    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:35.753801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753718    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2587439-0cd9-4318-bb39-4a1c85b405fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:35.753801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753751    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.754020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753800    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-data\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.754020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753831    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-credential-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.754020 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.753801    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:35.755461 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.755435    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:35.755681 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.755663    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-kube-api-access-6xzgx" (OuterVolumeSpecName: "kube-api-access-6xzgx") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "kube-api-access-6xzgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:35.755759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.755675    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:38:35.755759 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.755718    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2587439-0cd9-4318-bb39-4a1c85b405fc" (UID: "c2587439-0cd9-4318-bb39-4a1c85b405fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:38:35.854478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.854444    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2587439-0cd9-4318-bb39-4a1c85b405fc-istiod-ca-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.854478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.854473    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-podinfo\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.854478 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.854482    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-envoy\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.854741 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.854490    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-istio-token\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.854741 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.854499    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xzgx\" (UniqueName: \"kubernetes.io/projected/c2587439-0cd9-4318-bb39-4a1c85b405fc-kube-api-access-6xzgx\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:35.854741 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:35.854531    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2587439-0cd9-4318-bb39-4a1c85b405fc-workload-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:36.506928 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.506893    2583 generic.go:356] "Generic (PLEG): container finished" podID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerID="a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a" exitCode=0
Sep 10 21:38:36.507100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.506954    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"
Sep 10 21:38:36.507100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.506968    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" event={"ID":"c2587439-0cd9-4318-bb39-4a1c85b405fc","Type":"ContainerDied","Data":"a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a"}
Sep 10 21:38:36.507100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.507046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65" event={"ID":"c2587439-0cd9-4318-bb39-4a1c85b405fc","Type":"ContainerDied","Data":"2dadbd26b614bdf735d3d973693fc17003aae8c3316567cab30d0bf4108cf242"}
Sep 10 21:38:36.507100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.507068    2583 scope.go:117] "RemoveContainer" containerID="a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a"
Sep 10 21:38:36.515948 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.515789    2583 scope.go:117] "RemoveContainer" containerID="5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180"
Sep 10 21:38:36.524285 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.524220    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"]
Sep 10 21:38:36.525337 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.525303    2583 scope.go:117] "RemoveContainer" containerID="a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a"
Sep 10 21:38:36.525644 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:38:36.525625    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a\": container with ID starting with a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a not found: ID does not exist" containerID="a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a"
Sep 10 21:38:36.525713 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.525653    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a"} err="failed to get container status \"a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a\": rpc error: code = NotFound desc = could not find container \"a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a\": container with ID starting with a24d14ad7c892f44b306b48444a282d26553e7b463c086cb51c6f72d95fb409a not found: ID does not exist"
Sep 10 21:38:36.525713 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.525672    2583 scope.go:117] "RemoveContainer" containerID="5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180"
Sep 10 21:38:36.525940 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:38:36.525921    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180\": container with ID starting with 5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180 not found: ID does not exist" containerID="5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180"
Sep 10 21:38:36.525994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.525944    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180"} err="failed to get container status \"5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180\": rpc error: code = NotFound desc = could not find container \"5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180\": container with ID starting with 5d0b86180cf60b8413b6addaabcb1e3a40360dcec9735ec7436c14b3c91de180 not found: ID does not exist"
Sep 10 21:38:36.526462 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:36.526445    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-c8d758-openshift-default-7bc6cf68f6-rhm65"]
Sep 10 21:38:37.961461 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:37.961430    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" path="/var/lib/kubelet/pods/c2587439-0cd9-4318-bb39-4a1c85b405fc/volumes"
Sep 10 21:38:41.260979 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.260948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:41.291784 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.291749    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"]
Sep 10 21:38:41.292337 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.292305    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt" podUID="2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" containerName="payload-processing" containerID="cri-o://3d12eae0d3bacf4950a6af87594c7786f507a2c58ca5cff25e4ff42fa9f07ba2" gracePeriod=30
Sep 10 21:38:41.299068 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.299033    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"]
Sep 10 21:38:41.299449 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.299237    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7" podUID="54b75b35-a805-444e-b3b1-e02b161fc2eb" containerName="payload-pre-processing" containerID="cri-o://6a379af6d69110385c5221f1c36aec987446bc9929a3b05be34406442c0e5f3c" gracePeriod=30
Sep 10 21:38:41.328655 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.328628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:41.391222 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.391193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:41.467281 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.467249    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"]
Sep 10 21:38:41.470662 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.470636    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f25ef-sj7mn"]
Sep 10 21:38:41.471043 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.471016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:41.532755 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.532657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:41.537388 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.537363    2583 generic.go:356] "Generic (PLEG): container finished" podID="54b75b35-a805-444e-b3b1-e02b161fc2eb" containerID="6a379af6d69110385c5221f1c36aec987446bc9929a3b05be34406442c0e5f3c" exitCode=0
Sep 10 21:38:41.537544 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.537439    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7" event={"ID":"54b75b35-a805-444e-b3b1-e02b161fc2eb","Type":"ContainerDied","Data":"6a379af6d69110385c5221f1c36aec987446bc9929a3b05be34406442c0e5f3c"}
Sep 10 21:38:41.539282 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.539259    2583 generic.go:356] "Generic (PLEG): container finished" podID="2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" containerID="3d12eae0d3bacf4950a6af87594c7786f507a2c58ca5cff25e4ff42fa9f07ba2" exitCode=0
Sep 10 21:38:41.539385 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.539313    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt" event={"ID":"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364","Type":"ContainerDied","Data":"3d12eae0d3bacf4950a6af87594c7786f507a2c58ca5cff25e4ff42fa9f07ba2"}
Sep 10 21:38:41.579377 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.579356    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:38:41.582449 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.582432    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:38:41.717012 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.716973    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nrst\" (UniqueName: \"kubernetes.io/projected/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-kube-api-access-8nrst\") pod \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") "
Sep 10 21:38:41.717216 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.717078    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-plugins-config-volume\") pod \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\" (UID: \"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364\") "
Sep 10 21:38:41.717216 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.717145    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7rts\" (UniqueName: \"kubernetes.io/projected/54b75b35-a805-444e-b3b1-e02b161fc2eb-kube-api-access-j7rts\") pod \"54b75b35-a805-444e-b3b1-e02b161fc2eb\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") "
Sep 10 21:38:41.717216 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.717162    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b75b35-a805-444e-b3b1-e02b161fc2eb-plugins-config-volume\") pod \"54b75b35-a805-444e-b3b1-e02b161fc2eb\" (UID: \"54b75b35-a805-444e-b3b1-e02b161fc2eb\") "
Sep 10 21:38:41.717559 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.717525    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" (UID: "2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:41.717657 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.717624    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54b75b35-a805-444e-b3b1-e02b161fc2eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54b75b35-a805-444e-b3b1-e02b161fc2eb" (UID: "54b75b35-a805-444e-b3b1-e02b161fc2eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:41.719166 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.719129    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-kube-api-access-8nrst" (OuterVolumeSpecName: "kube-api-access-8nrst") pod "2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" (UID: "2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364"). InnerVolumeSpecName "kube-api-access-8nrst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:41.719264 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.719231    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54b75b35-a805-444e-b3b1-e02b161fc2eb-kube-api-access-j7rts" (OuterVolumeSpecName: "kube-api-access-j7rts") pod "54b75b35-a805-444e-b3b1-e02b161fc2eb" (UID: "54b75b35-a805-444e-b3b1-e02b161fc2eb"). InnerVolumeSpecName "kube-api-access-j7rts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:41.818791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.818704    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:41.818791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.818734    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7rts\" (UniqueName: \"kubernetes.io/projected/54b75b35-a805-444e-b3b1-e02b161fc2eb-kube-api-access-j7rts\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:41.818791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.818744    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b75b35-a805-444e-b3b1-e02b161fc2eb-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:41.818791 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.818753    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nrst\" (UniqueName: \"kubernetes.io/projected/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364-kube-api-access-8nrst\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:41.863146 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.863111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:41.960257 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:41.960225    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7de44e8e-3431-4134-98d0-1a782acad777" path="/var/lib/kubelet/pods/7de44e8e-3431-4134-98d0-1a782acad777/volumes"
Sep 10 21:38:42.545868 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.545839    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"
Sep 10 21:38:42.546303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.545875    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7" event={"ID":"54b75b35-a805-444e-b3b1-e02b161fc2eb","Type":"ContainerDied","Data":"cb1f3b7b3b0639aa858b7c168d7b2fb5082535b913e98bb6c0c3a35af8074eaa"}
Sep 10 21:38:42.546303 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.545945    2583 scope.go:117] "RemoveContainer" containerID="6a379af6d69110385c5221f1c36aec987446bc9929a3b05be34406442c0e5f3c"
Sep 10 21:38:42.547429 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.547412    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"
Sep 10 21:38:42.547546 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.547444    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt" event={"ID":"2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364","Type":"ContainerDied","Data":"c5932927f5c4af030ec3cb1d1df0dcb9e663bc514521e4270406ad6f63817bf4"}
Sep 10 21:38:42.555165 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.555147    2583 scope.go:117] "RemoveContainer" containerID="3d12eae0d3bacf4950a6af87594c7786f507a2c58ca5cff25e4ff42fa9f07ba2"
Sep 10 21:38:42.562286 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.562223    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"]
Sep 10 21:38:42.566343 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.566318    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5f25ef-df45488bb-hf2c7"]
Sep 10 21:38:42.574129 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.574103    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"]
Sep 10 21:38:42.576545 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:42.576524    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5f25ef-6f54f8fdbf-v72bt"]
Sep 10 21:38:43.960804 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:43.960768    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" path="/var/lib/kubelet/pods/2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364/volumes"
Sep 10 21:38:43.961191 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:43.961176    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54b75b35-a805-444e-b3b1-e02b161fc2eb" path="/var/lib/kubelet/pods/54b75b35-a805-444e-b3b1-e02b161fc2eb/volumes"
Sep 10 21:38:47.612553 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:47.612498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:47.712843 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:47.712811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:47.864732 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:47.864643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:48.300176 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:48.300131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:58.683080 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.683039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:58.683597 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.683395    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"]
Sep 10 21:38:58.688401 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.688376    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-36b938-29817930-24t4h"]
Sep 10 21:38:58.718737 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.718704    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"]
Sep 10 21:38:58.718961 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.718934    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb" podUID="4848ca30-a444-4635-9a5c-0680aefb235a" containerName="payload-pre-processing" containerID="cri-o://250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6" gracePeriod=30
Sep 10 21:38:58.782075 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.782041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:58.880492 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.880448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:58.968270 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.968237    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:38:58.983473 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:58.983432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:59.084099 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.084068    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4848ca30-a444-4635-9a5c-0680aefb235a-plugins-config-volume\") pod \"4848ca30-a444-4635-9a5c-0680aefb235a\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") "
Sep 10 21:38:59.084320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.084228    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72dxp\" (UniqueName: \"kubernetes.io/projected/4848ca30-a444-4635-9a5c-0680aefb235a-kube-api-access-72dxp\") pod \"4848ca30-a444-4635-9a5c-0680aefb235a\" (UID: \"4848ca30-a444-4635-9a5c-0680aefb235a\") "
Sep 10 21:38:59.084452 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.084429    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4848ca30-a444-4635-9a5c-0680aefb235a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4848ca30-a444-4635-9a5c-0680aefb235a" (UID: "4848ca30-a444-4635-9a5c-0680aefb235a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:38:59.084595 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.084577    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4848ca30-a444-4635-9a5c-0680aefb235a-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.086283 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.086265    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4848ca30-a444-4635-9a5c-0680aefb235a-kube-api-access-72dxp" (OuterVolumeSpecName: "kube-api-access-72dxp") pod "4848ca30-a444-4635-9a5c-0680aefb235a" (UID: "4848ca30-a444-4635-9a5c-0680aefb235a"). InnerVolumeSpecName "kube-api-access-72dxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:38:59.185963 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.185926    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72dxp\" (UniqueName: \"kubernetes.io/projected/4848ca30-a444-4635-9a5c-0680aefb235a-kube-api-access-72dxp\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:38:59.258387 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.258288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:59.643614 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.643579    2583 generic.go:356] "Generic (PLEG): container finished" podID="4848ca30-a444-4635-9a5c-0680aefb235a" containerID="250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6" exitCode=0
Sep 10 21:38:59.643814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.643624    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"
Sep 10 21:38:59.643814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.643651    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb" event={"ID":"4848ca30-a444-4635-9a5c-0680aefb235a","Type":"ContainerDied","Data":"250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6"}
Sep 10 21:38:59.643814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.643703    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb" event={"ID":"4848ca30-a444-4635-9a5c-0680aefb235a","Type":"ContainerDied","Data":"a8a15c948b026c370bf12326078b3b10e60ab80df5a4db984466191ae3add56f"}
Sep 10 21:38:59.643814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.643734    2583 scope.go:117] "RemoveContainer" containerID="250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6"
Sep 10 21:38:59.652341 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.652325    2583 scope.go:117] "RemoveContainer" containerID="250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6"
Sep 10 21:38:59.652627 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:38:59.652608    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6\": container with ID starting with 250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6 not found: ID does not exist" containerID="250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6"
Sep 10 21:38:59.652676 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.652637    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6"} err="failed to get container status \"250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6\": rpc error: code = NotFound desc = could not find container \"250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6\": container with ID starting with 250a68c2a79f0ae4360e83da1d946192fc6a357d9fdcd7e9b00978afa168bde6 not found: ID does not exist"
Sep 10 21:38:59.662878 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.662848    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"]
Sep 10 21:38:59.666051 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.666028    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36b938-79ffbcf99d-7p9rb"]
Sep 10 21:38:59.960515 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.960416    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4848ca30-a444-4635-9a5c-0680aefb235a" path="/var/lib/kubelet/pods/4848ca30-a444-4635-9a5c-0680aefb235a/volumes"
Sep 10 21:38:59.960888 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:38:59.960750    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dac1895a-72df-4f98-8709-2e923eaebacb" path="/var/lib/kubelet/pods/dac1895a-72df-4f98-8709-2e923eaebacb/volumes"
Sep 10 21:39:05.025362 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:05.025323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:05.090053 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:05.090004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:05.250185 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:05.250150    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"]
Sep 10 21:39:05.250394 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:05.250370    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" containerID="cri-o://b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382" gracePeriod=30
Sep 10 21:39:05.270769 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:05.270726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:05.790694 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:05.790658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:07.655121 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:07.655074    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:39:07.655541 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:07.655162    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:39:10.232684 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.232649    2583 scope.go:117] "RemoveContainer" containerID="a0d8677377166d789b9c2130312de907d8896dc7f2e37f746b1e758eaa179b6c"
Sep 10 21:39:10.510314 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.510286    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:39:10.595110 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595075    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-token\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595110 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595116    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-certs\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595144    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-socket\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595160    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-credential-socket\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595184    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-envoy\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595219    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pwbj\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-kube-api-access-9pwbj\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595256    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb99666-8595-4571-adb3-c0b63989a8e8-istiod-ca-cert\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595298    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-data\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595386    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-podinfo\") pod \"3fb99666-8595-4571-adb3-c0b63989a8e8\" (UID: \"3fb99666-8595-4571-adb3-c0b63989a8e8\") "
Sep 10 21:39:10.595708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595437    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:10.595708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595556    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:10.595708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595447    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:10.595708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595700    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:10.595958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595756    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.595958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595770    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-workload-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.595958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595785    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-credential-socket\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.595958 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.595784    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fb99666-8595-4571-adb3-c0b63989a8e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:39:10.597599 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.597574    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:39:10.597708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.597575    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:10.597708 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.597625    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-kube-api-access-9pwbj" (OuterVolumeSpecName: "kube-api-access-9pwbj") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "kube-api-access-9pwbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:10.597815 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.597777    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3fb99666-8595-4571-adb3-c0b63989a8e8" (UID: "3fb99666-8595-4571-adb3-c0b63989a8e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:39:10.696942 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.696900    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pwbj\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-kube-api-access-9pwbj\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.696942 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.696938    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb99666-8595-4571-adb3-c0b63989a8e8-istiod-ca-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.697147 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.696953    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-data\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.697147 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.696966    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-podinfo\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.697147 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.696978    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-token\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.697147 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.696991    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb99666-8595-4571-adb3-c0b63989a8e8-istio-envoy\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:10.702550 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.702525    2583 generic.go:356] "Generic (PLEG): container finished" podID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerID="b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382" exitCode=0
Sep 10 21:39:10.702665 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.702630    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"
Sep 10 21:39:10.702665 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.702648    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" event={"ID":"3fb99666-8595-4571-adb3-c0b63989a8e8","Type":"ContainerDied","Data":"b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382"}
Sep 10 21:39:10.702736 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.702691    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx" event={"ID":"3fb99666-8595-4571-adb3-c0b63989a8e8","Type":"ContainerDied","Data":"70aad36aed73f838863f3d0f032d71a4777fa48d415014780f10663e014c8f96"}
Sep 10 21:39:10.702736 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.702718    2583 scope.go:117] "RemoveContainer" containerID="b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382"
Sep 10 21:39:10.711807 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.711790    2583 scope.go:117] "RemoveContainer" containerID="6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b"
Sep 10 21:39:10.719597 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.719579    2583 scope.go:117] "RemoveContainer" containerID="b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382"
Sep 10 21:39:10.719847 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:39:10.719830    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382\": container with ID starting with b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382 not found: ID does not exist" containerID="b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382"
Sep 10 21:39:10.719897 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.719854    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382"} err="failed to get container status \"b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382\": rpc error: code = NotFound desc = could not find container \"b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382\": container with ID starting with b8a03922cd116481d12176bc2da78346e312ee955c658f0780b43b59d8a1c382 not found: ID does not exist"
Sep 10 21:39:10.719897 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.719869    2583 scope.go:117] "RemoveContainer" containerID="6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b"
Sep 10 21:39:10.720113 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:39:10.720097    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b\": container with ID starting with 6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b not found: ID does not exist" containerID="6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b"
Sep 10 21:39:10.720168 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.720116    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b"} err="failed to get container status \"6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b\": rpc error: code = NotFound desc = could not find container \"6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b\": container with ID starting with 6be497ece49cb950766696cd729682c792a4b07569d86f144fd48dd70153632b not found: ID does not exist"
Sep 10 21:39:10.725854 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.724569    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"]
Sep 10 21:39:10.731908 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:10.731885    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-36b938-openshift-default-6cd65ff4c8-67cnx"]
Sep 10 21:39:11.961373 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:11.961336    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" path="/var/lib/kubelet/pods/3fb99666-8595-4571-adb3-c0b63989a8e8/volumes"
Sep 10 21:39:16.164531 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.164457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:16.193076 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.193041    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"]
Sep 10 21:39:16.193342 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.193308    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx" podUID="73bb8431-e77c-41c1-8f63-ccfab23bc730" containerName="payload-processing" containerID="cri-o://feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4" gracePeriod=30
Sep 10 21:39:16.263947 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.263895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:16.358877 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.358457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:16.430375 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.430024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:16.458885 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.458858    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:39:16.549377 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.549341    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7skk\" (UniqueName: \"kubernetes.io/projected/73bb8431-e77c-41c1-8f63-ccfab23bc730-kube-api-access-l7skk\") pod \"73bb8431-e77c-41c1-8f63-ccfab23bc730\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") "
Sep 10 21:39:16.549581 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.549405    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bb8431-e77c-41c1-8f63-ccfab23bc730-plugins-config-volume\") pod \"73bb8431-e77c-41c1-8f63-ccfab23bc730\" (UID: \"73bb8431-e77c-41c1-8f63-ccfab23bc730\") "
Sep 10 21:39:16.549821 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.549796    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73bb8431-e77c-41c1-8f63-ccfab23bc730-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73bb8431-e77c-41c1-8f63-ccfab23bc730" (UID: "73bb8431-e77c-41c1-8f63-ccfab23bc730"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:39:16.551558 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.551534    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73bb8431-e77c-41c1-8f63-ccfab23bc730-kube-api-access-l7skk" (OuterVolumeSpecName: "kube-api-access-l7skk") pod "73bb8431-e77c-41c1-8f63-ccfab23bc730" (UID: "73bb8431-e77c-41c1-8f63-ccfab23bc730"). InnerVolumeSpecName "kube-api-access-l7skk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:16.650534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.650453    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7skk\" (UniqueName: \"kubernetes.io/projected/73bb8431-e77c-41c1-8f63-ccfab23bc730-kube-api-access-l7skk\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.650534 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.650497    2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bb8431-e77c-41c1-8f63-ccfab23bc730-plugins-config-volume\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:16.699179 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.699089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:16.735489 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.735454    2583 generic.go:356] "Generic (PLEG): container finished" podID="73bb8431-e77c-41c1-8f63-ccfab23bc730" containerID="feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4" exitCode=0
Sep 10 21:39:16.735664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.735514    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"
Sep 10 21:39:16.735664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.735547    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx" event={"ID":"73bb8431-e77c-41c1-8f63-ccfab23bc730","Type":"ContainerDied","Data":"feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4"}
Sep 10 21:39:16.735664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.735606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx" event={"ID":"73bb8431-e77c-41c1-8f63-ccfab23bc730","Type":"ContainerDied","Data":"b592b149fe70238a948da407504ffd9ed7953dfb5593135760c08d8b990b001c"}
Sep 10 21:39:16.735664 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.735636    2583 scope.go:117] "RemoveContainer" containerID="feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4"
Sep 10 21:39:16.745371 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.745349    2583 scope.go:117] "RemoveContainer" containerID="feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4"
Sep 10 21:39:16.745714 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:39:16.745695    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4\": container with ID starting with feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4 not found: ID does not exist" containerID="feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4"
Sep 10 21:39:16.745801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.745726    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4"} err="failed to get container status \"feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4\": rpc error: code = NotFound desc = could not find container \"feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4\": container with ID starting with feafd1b97ef9789c98b4660592e5f313d466387165a62ca6871e4b53c3934ae4 not found: ID does not exist"
Sep 10 21:39:16.755647 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.755618    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"]
Sep 10 21:39:16.757364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:16.757343    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-1f3879-68c475759c-48bfx"]
Sep 10 21:39:17.961365 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:17.961320    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73bb8431-e77c-41c1-8f63-ccfab23bc730" path="/var/lib/kubelet/pods/73bb8431-e77c-41c1-8f63-ccfab23bc730/volumes"
Sep 10 21:39:22.449633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:22.449583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:22.503533 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:22.503483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:22.712700 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:22.712598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:25.148014 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.147974    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-d9djh"]
Sep 10 21:39:25.148462 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.148208    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77cc466dd6-d9djh" podUID="1bc46f35-6c0b-4406-9578-b94c0a4816c3" containerName="manager" containerID="cri-o://7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc" gracePeriod=30
Sep 10 21:39:25.386011 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.385986    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:39:25.536033 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.535945    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnv4p\" (UniqueName: \"kubernetes.io/projected/1bc46f35-6c0b-4406-9578-b94c0a4816c3-kube-api-access-hnv4p\") pod \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") "
Sep 10 21:39:25.536033 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.535994    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-webhook-cert\") pod \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") "
Sep 10 21:39:25.536033 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.536013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-metrics-certs\") pod \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\" (UID: \"1bc46f35-6c0b-4406-9578-b94c0a4816c3\") "
Sep 10 21:39:25.538107 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.538073    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1bc46f35-6c0b-4406-9578-b94c0a4816c3" (UID: "1bc46f35-6c0b-4406-9578-b94c0a4816c3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:39:25.538608 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.538590    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1bc46f35-6c0b-4406-9578-b94c0a4816c3" (UID: "1bc46f35-6c0b-4406-9578-b94c0a4816c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:39:25.538698 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.538624    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc46f35-6c0b-4406-9578-b94c0a4816c3-kube-api-access-hnv4p" (OuterVolumeSpecName: "kube-api-access-hnv4p") pod "1bc46f35-6c0b-4406-9578-b94c0a4816c3" (UID: "1bc46f35-6c0b-4406-9578-b94c0a4816c3"). InnerVolumeSpecName "kube-api-access-hnv4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:39:25.637266 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.637225    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnv4p\" (UniqueName: \"kubernetes.io/projected/1bc46f35-6c0b-4406-9578-b94c0a4816c3-kube-api-access-hnv4p\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:25.637266 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.637259    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-webhook-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:25.637266 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.637274    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc46f35-6c0b-4406-9578-b94c0a4816c3-metrics-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:39:25.786570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.786462    2583 generic.go:356] "Generic (PLEG): container finished" podID="1bc46f35-6c0b-4406-9578-b94c0a4816c3" containerID="7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc" exitCode=0
Sep 10 21:39:25.786570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.786526    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-d9djh"
Sep 10 21:39:25.786570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.786559    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-d9djh" event={"ID":"1bc46f35-6c0b-4406-9578-b94c0a4816c3","Type":"ContainerDied","Data":"7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc"}
Sep 10 21:39:25.786792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.786618    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-d9djh" event={"ID":"1bc46f35-6c0b-4406-9578-b94c0a4816c3","Type":"ContainerDied","Data":"03efd64eac91621906ce11b1f2b18b3ef96b2952a20650319a37d6ab9b104dc7"}
Sep 10 21:39:25.786792 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.786640    2583 scope.go:117] "RemoveContainer" containerID="7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc"
Sep 10 21:39:25.796488 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.796469    2583 scope.go:117] "RemoveContainer" containerID="7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc"
Sep 10 21:39:25.796966 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:39:25.796945    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc\": container with ID starting with 7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc not found: ID does not exist" containerID="7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc"
Sep 10 21:39:25.797016 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.796974    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc"} err="failed to get container status \"7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc\": rpc error: code = NotFound desc = could not find container \"7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc\": container with ID starting with 7d69334ca5d8dd60fe6f6517829da0b758a0def805616a57a7e1d53e9061f1dc not found: ID does not exist"
Sep 10 21:39:25.806037 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.806015    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-d9djh"]
Sep 10 21:39:25.811694 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.811675    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-d9djh"]
Sep 10 21:39:25.960465 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:39:25.960434    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bc46f35-6c0b-4406-9578-b94c0a4816c3" path="/var/lib/kubelet/pods/1bc46f35-6c0b-4406-9578-b94c0a4816c3/volumes"
Sep 10 21:40:10.058324 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:10.058294    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:40:10.060100 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:10.060080    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:40:26.596096 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596055    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77cc466dd6-zwbc5"]
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596444    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596458    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596468    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596474    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596482    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" containerName="payload-processing"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596488    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" containerName="payload-processing"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596499    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596521    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596532    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4848ca30-a444-4635-9a5c-0680aefb235a" containerName="payload-pre-processing"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596537    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4848ca30-a444-4635-9a5c-0680aefb235a" containerName="payload-pre-processing"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596543    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73bb8431-e77c-41c1-8f63-ccfab23bc730" containerName="payload-processing"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596548    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="73bb8431-e77c-41c1-8f63-ccfab23bc730" containerName="payload-processing"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596554    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de44e8e-3431-4134-98d0-1a782acad777" containerName="revoke-keys"
Sep 10 21:40:26.596560 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596559    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de44e8e-3431-4134-98d0-1a782acad777" containerName="revoke-keys"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596571    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54b75b35-a805-444e-b3b1-e02b161fc2eb" containerName="payload-pre-processing"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596577    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="54b75b35-a805-444e-b3b1-e02b161fc2eb" containerName="payload-pre-processing"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596587    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bc46f35-6c0b-4406-9578-b94c0a4816c3" containerName="manager"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596595    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc46f35-6c0b-4406-9578-b94c0a4816c3" containerName="manager"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596652    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596658    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aca76a4-8c91-4bd2-ad3d-cd9a4fb97364" containerName="payload-processing"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596668    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596677    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596684    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4848ca30-a444-4635-9a5c-0680aefb235a" containerName="payload-pre-processing"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596691    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="54b75b35-a805-444e-b3b1-e02b161fc2eb" containerName="payload-pre-processing"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596696    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de44e8e-3431-4134-98d0-1a782acad777" containerName="revoke-keys"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596703    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="73bb8431-e77c-41c1-8f63-ccfab23bc730" containerName="payload-processing"
Sep 10 21:40:26.596994 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.596708    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bc46f35-6c0b-4406-9578-b94c0a4816c3" containerName="manager"
Sep 10 21:40:26.599841 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.599824    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.601927 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.601903    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 21:40:26.602102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.602024    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 21:40:26.602102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.602043    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 21:40:26.602102 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.602050    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mxbbt\""
Sep 10 21:40:26.609839 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.609807    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-zwbc5"]
Sep 10 21:40:26.699951 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.699915    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-webhook-cert\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.699951 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.699952    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb6tk\" (UniqueName: \"kubernetes.io/projected/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-kube-api-access-mb6tk\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.700177 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.700105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-metrics-certs\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.800884 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.800847    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-metrics-certs\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.801034 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.800932    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-webhook-cert\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.801034 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.800952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb6tk\" (UniqueName: \"kubernetes.io/projected/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-kube-api-access-mb6tk\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.803347 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.803316    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-webhook-cert\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.803347 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.803341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-metrics-certs\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.808273 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.808256    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb6tk\" (UniqueName: \"kubernetes.io/projected/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-kube-api-access-mb6tk\") pod \"maas-controller-77cc466dd6-zwbc5\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") " pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:26.911257 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:26.911221    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:27.180414 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:27.180336    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-zwbc5"]
Sep 10 21:40:27.182986 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:40:27.182960    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6573308d_abee_4b2a_a426_a0fe1aaa7c5b.slice/crio-fe8b33a0eb98ee655979b4247463bab4a9f2e5b67a00f9dcb49c51934dd15433 WatchSource:0}: Error finding container fe8b33a0eb98ee655979b4247463bab4a9f2e5b67a00f9dcb49c51934dd15433: Status 404 returned error can't find the container with id fe8b33a0eb98ee655979b4247463bab4a9f2e5b67a00f9dcb49c51934dd15433
Sep 10 21:40:27.184324 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:27.184306    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:40:28.101063 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:28.101021    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-zwbc5" event={"ID":"6573308d-abee-4b2a-a426-a0fe1aaa7c5b","Type":"ContainerStarted","Data":"077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61"}
Sep 10 21:40:28.101483 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:28.101104    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-zwbc5" event={"ID":"6573308d-abee-4b2a-a426-a0fe1aaa7c5b","Type":"ContainerStarted","Data":"fe8b33a0eb98ee655979b4247463bab4a9f2e5b67a00f9dcb49c51934dd15433"}
Sep 10 21:40:28.101483 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:28.101168    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:28.116770 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:28.116611    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77cc466dd6-zwbc5" podStartSLOduration=1.691260862 podStartE2EDuration="2.116591706s" totalImagesPullingTime="425.330844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:40:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:40:28.115712741 +0000 UTC m=+1818.642677088" watchObservedRunningTime="2026-09-10 21:40:28.116591706 +0000 UTC m=+1818.643556084"
Sep 10 21:40:39.108391 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:39.108355    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:47.165157 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:47.165121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:48.310627 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.310591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:48.407797 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.407764    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-zwbc5"]
Sep 10 21:40:48.408045 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.408023    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-77cc466dd6-zwbc5" podUID="6573308d-abee-4b2a-a426-a0fe1aaa7c5b" containerName="manager" containerID="cri-o://077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61" gracePeriod=30
Sep 10 21:40:48.659921 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.659881    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:48.802610 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.802571    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-webhook-cert\") pod \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") "
Sep 10 21:40:48.802814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.802623    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-metrics-certs\") pod \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") "
Sep 10 21:40:48.802814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.802651    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb6tk\" (UniqueName: \"kubernetes.io/projected/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-kube-api-access-mb6tk\") pod \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\" (UID: \"6573308d-abee-4b2a-a426-a0fe1aaa7c5b\") "
Sep 10 21:40:48.804961 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.804920    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-kube-api-access-mb6tk" (OuterVolumeSpecName: "kube-api-access-mb6tk") pod "6573308d-abee-4b2a-a426-a0fe1aaa7c5b" (UID: "6573308d-abee-4b2a-a426-a0fe1aaa7c5b"). InnerVolumeSpecName "kube-api-access-mb6tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:40:48.805092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.805062    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6573308d-abee-4b2a-a426-a0fe1aaa7c5b" (UID: "6573308d-abee-4b2a-a426-a0fe1aaa7c5b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:40:48.805156 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.805113    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6573308d-abee-4b2a-a426-a0fe1aaa7c5b" (UID: "6573308d-abee-4b2a-a426-a0fe1aaa7c5b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:40:48.903894 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.903854    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-webhook-cert\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:40:48.904062 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.903899    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-metrics-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:40:48.904062 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:48.903915    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb6tk\" (UniqueName: \"kubernetes.io/projected/6573308d-abee-4b2a-a426-a0fe1aaa7c5b-kube-api-access-mb6tk\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:40:49.202379 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.202282    2583 generic.go:356] "Generic (PLEG): container finished" podID="6573308d-abee-4b2a-a426-a0fe1aaa7c5b" containerID="077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61" exitCode=0
Sep 10 21:40:49.202379 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.202333    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-zwbc5"
Sep 10 21:40:49.202379 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.202344    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-zwbc5" event={"ID":"6573308d-abee-4b2a-a426-a0fe1aaa7c5b","Type":"ContainerDied","Data":"077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61"}
Sep 10 21:40:49.202860 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.202393    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-zwbc5" event={"ID":"6573308d-abee-4b2a-a426-a0fe1aaa7c5b","Type":"ContainerDied","Data":"fe8b33a0eb98ee655979b4247463bab4a9f2e5b67a00f9dcb49c51934dd15433"}
Sep 10 21:40:49.202860 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.202413    2583 scope.go:117] "RemoveContainer" containerID="077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61"
Sep 10 21:40:49.211550 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.211522    2583 scope.go:117] "RemoveContainer" containerID="077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61"
Sep 10 21:40:49.211837 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:40:49.211816    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61\": container with ID starting with 077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61 not found: ID does not exist" containerID="077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61"
Sep 10 21:40:49.211900 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.211844    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61"} err="failed to get container status \"077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61\": rpc error: code = NotFound desc = could not find container \"077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61\": container with ID starting with 077b44efc8c1ee92f9ec48c986552819fc535ac86cae877be77db5083fff7a61 not found: ID does not exist"
Sep 10 21:40:49.221987 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.221959    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-zwbc5"]
Sep 10 21:40:49.225848 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.225824    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-zwbc5"]
Sep 10 21:40:49.866245 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.866211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:49.960939 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.960897    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6573308d-abee-4b2a-a426-a0fe1aaa7c5b" path="/var/lib/kubelet/pods/6573308d-abee-4b2a-a426-a0fe1aaa7c5b/volumes"
Sep 10 21:40:49.976171 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.976132    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-77cc466dd6-bngq7"]
Sep 10 21:40:49.976482 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.976469    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy"
Sep 10 21:40:49.976546 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.976484    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2587439-0cd9-4318-bb39-4a1c85b405fc" containerName="istio-proxy"
Sep 10 21:40:49.976546 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.976519    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6573308d-abee-4b2a-a426-a0fe1aaa7c5b" containerName="manager"
Sep 10 21:40:49.976546 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.976526    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6573308d-abee-4b2a-a426-a0fe1aaa7c5b" containerName="manager"
Sep 10 21:40:49.976649 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.976596    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fb99666-8595-4571-adb3-c0b63989a8e8" containerName="istio-proxy"
Sep 10 21:40:49.976649 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.976604    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6573308d-abee-4b2a-a426-a0fe1aaa7c5b" containerName="manager"
Sep 10 21:40:49.980756 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.980735    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:49.982727 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.982706    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 21:40:49.982855 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.982819    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 21:40:49.982922 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.982883    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mxbbt\""
Sep 10 21:40:49.982922 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.982896    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 21:40:49.989726 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:49.989702    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-bngq7"]
Sep 10 21:40:50.115710 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.115676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5e0ce67-7930-4da4-9273-143093bdb094-metrics-certs\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.115888 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.115749    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5e0ce67-7930-4da4-9273-143093bdb094-webhook-cert\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.115888 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.115782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnmf2\" (UniqueName: \"kubernetes.io/projected/e5e0ce67-7930-4da4-9273-143093bdb094-kube-api-access-hnmf2\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.217022 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.216989    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5e0ce67-7930-4da4-9273-143093bdb094-webhook-cert\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.217022 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.217027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnmf2\" (UniqueName: \"kubernetes.io/projected/e5e0ce67-7930-4da4-9273-143093bdb094-kube-api-access-hnmf2\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.217243 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.217073    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5e0ce67-7930-4da4-9273-143093bdb094-metrics-certs\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.219692 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.219669    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5e0ce67-7930-4da4-9273-143093bdb094-metrics-certs\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.219775 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.219669    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5e0ce67-7930-4da4-9273-143093bdb094-webhook-cert\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.224544 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.224519    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnmf2\" (UniqueName: \"kubernetes.io/projected/e5e0ce67-7930-4da4-9273-143093bdb094-kube-api-access-hnmf2\") pod \"maas-controller-77cc466dd6-bngq7\" (UID: \"e5e0ce67-7930-4da4-9273-143093bdb094\") " pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.292117 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.292088    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:50.579665 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:50.579639    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-77cc466dd6-bngq7"]
Sep 10 21:40:50.582349 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:40:50.582320    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e0ce67_7930_4da4_9273_143093bdb094.slice/crio-7abc8ef1eca6e1311fddfbbafe7261faf8bc17416e0bf91c2b1e88c42b942711 WatchSource:0}: Error finding container 7abc8ef1eca6e1311fddfbbafe7261faf8bc17416e0bf91c2b1e88c42b942711: Status 404 returned error can't find the container with id 7abc8ef1eca6e1311fddfbbafe7261faf8bc17416e0bf91c2b1e88c42b942711
Sep 10 21:40:51.213926 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:51.213895    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-bngq7" event={"ID":"e5e0ce67-7930-4da4-9273-143093bdb094","Type":"ContainerStarted","Data":"28c50789e752d2b1edcf1bcf808c5c76058c1f0ec79a974e5e0a939f3b9e99c7"}
Sep 10 21:40:51.214296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:51.213946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-77cc466dd6-bngq7" event={"ID":"e5e0ce67-7930-4da4-9273-143093bdb094","Type":"ContainerStarted","Data":"7abc8ef1eca6e1311fddfbbafe7261faf8bc17416e0bf91c2b1e88c42b942711"}
Sep 10 21:40:51.214296 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:51.214010    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:40:51.231041 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:40:51.230944    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-77cc466dd6-bngq7" podStartSLOduration=1.841848146 podStartE2EDuration="2.230928085s" totalImagesPullingTime="389.079939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:40:51.229311041 +0000 UTC m=+1841.756275389" watchObservedRunningTime="2026-09-10 21:40:51.230928085 +0000 UTC m=+1841.757892480"
Sep 10 21:41:02.220659 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:41:02.220620    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-77cc466dd6-bngq7"
Sep 10 21:41:05.201490 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:41:05.201450    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:10.288424 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:41:10.288395    2583 scope.go:117] "RemoveContainer" containerID="4fae12d55d24abc1c221028da21ce65cf48e2946e68e5175dc655e678a4b402a"
Sep 10 21:41:11.788413 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:41:11.788373    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:36.021023 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:43:36.020986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:46.155712 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:43:46.155676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:07.557814 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:07.557774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:08.312795 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:08.312747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:10.331871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:10.331843    2583 scope.go:117] "RemoveContainer" containerID="4f16dc8b472e022b4cf9e558b1558d1c3346cd1add297ac02c4d3ac5e35a61c3"
Sep 10 21:44:10.339871 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:10.339844    2583 scope.go:117] "RemoveContainer" containerID="ba9ca012b910a37eb42659975a56851ffa53ed44ecaa2f320cf05111c043a3a0"
Sep 10 21:44:28.308751 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.308717    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"]
Sep 10 21:44:28.311935 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.311919    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.323117 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.323080    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"]
Sep 10 21:44:28.422633 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.422601    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc09c37-a492-49d9-8437-2f13843584fd-maas-api-tls\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.422829 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.422652    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc09c37-a492-49d9-8437-2f13843584fd-metrics-certs\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.422829 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.422725    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kvk\" (UniqueName: \"kubernetes.io/projected/fbc09c37-a492-49d9-8437-2f13843584fd-kube-api-access-k8kvk\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.422829 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.422754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc09c37-a492-49d9-8437-2f13843584fd-openshift-service-ca\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.524119 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.524083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kvk\" (UniqueName: \"kubernetes.io/projected/fbc09c37-a492-49d9-8437-2f13843584fd-kube-api-access-k8kvk\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.524119 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.524120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc09c37-a492-49d9-8437-2f13843584fd-openshift-service-ca\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.524388 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.524169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc09c37-a492-49d9-8437-2f13843584fd-maas-api-tls\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.524388 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.524201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc09c37-a492-49d9-8437-2f13843584fd-metrics-certs\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.524854 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.524832    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc09c37-a492-49d9-8437-2f13843584fd-openshift-service-ca\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.526844 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.526821    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc09c37-a492-49d9-8437-2f13843584fd-metrics-certs\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.527013 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.526957    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc09c37-a492-49d9-8437-2f13843584fd-maas-api-tls\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.531552 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.531524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kvk\" (UniqueName: \"kubernetes.io/projected/fbc09c37-a492-49d9-8437-2f13843584fd-kube-api-access-k8kvk\") pod \"maas-api-84bdd77b86-t7n8p\" (UID: \"fbc09c37-a492-49d9-8437-2f13843584fd\") " pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.624742 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.624638    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:28.915876 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:28.915841    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"]
Sep 10 21:44:28.920695 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:44:28.920663    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc09c37_a492_49d9_8437_2f13843584fd.slice/crio-e00e0a6b8b60c498c2787c4949872b60dac00c01593e0d72a894f123da44a722 WatchSource:0}: Error finding container e00e0a6b8b60c498c2787c4949872b60dac00c01593e0d72a894f123da44a722: Status 404 returned error can't find the container with id e00e0a6b8b60c498c2787c4949872b60dac00c01593e0d72a894f123da44a722
Sep 10 21:44:29.263801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:29.263719    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p" event={"ID":"fbc09c37-a492-49d9-8437-2f13843584fd","Type":"ContainerStarted","Data":"e00e0a6b8b60c498c2787c4949872b60dac00c01593e0d72a894f123da44a722"}
Sep 10 21:44:30.269706 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:30.269674    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p" event={"ID":"fbc09c37-a492-49d9-8437-2f13843584fd","Type":"ContainerStarted","Data":"a9560506514c3ab363f6af6572618412d716f94c62d74b6b23e7022e67a5272f"}
Sep 10 21:44:30.270092 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:30.269750    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:30.287122 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:30.287071    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p" podStartSLOduration=1.8575258730000002 podStartE2EDuration="2.287052895s" totalImagesPullingTime="429.527022ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:44:30.286207695 +0000 UTC m=+2060.813172043" watchObservedRunningTime="2026-09-10 21:44:30.287052895 +0000 UTC m=+2060.814017242"
Sep 10 21:44:36.280325 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:36.280241    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84bdd77b86-t7n8p"
Sep 10 21:44:36.325127 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:36.325083    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"]
Sep 10 21:44:36.325373 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:36.325346    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q" podUID="12995527-11dd-40ed-b8af-c686dbd2d6fa" containerName="maas-api" containerID="cri-o://99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4" gracePeriod=30
Sep 10 21:44:37.569253 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.569231    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:44:37.710371 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.710291    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clqls\" (UniqueName: \"kubernetes.io/projected/12995527-11dd-40ed-b8af-c686dbd2d6fa-kube-api-access-clqls\") pod \"12995527-11dd-40ed-b8af-c686dbd2d6fa\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") "
Sep 10 21:44:37.710371 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.710344    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12995527-11dd-40ed-b8af-c686dbd2d6fa-openshift-service-ca\") pod \"12995527-11dd-40ed-b8af-c686dbd2d6fa\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") "
Sep 10 21:44:37.710648 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.710386    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs\") pod \"12995527-11dd-40ed-b8af-c686dbd2d6fa\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") "
Sep 10 21:44:37.710648 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.710437    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-maas-api-tls\") pod \"12995527-11dd-40ed-b8af-c686dbd2d6fa\" (UID: \"12995527-11dd-40ed-b8af-c686dbd2d6fa\") "
Sep 10 21:44:37.710906 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.710744    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12995527-11dd-40ed-b8af-c686dbd2d6fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "12995527-11dd-40ed-b8af-c686dbd2d6fa" (UID: "12995527-11dd-40ed-b8af-c686dbd2d6fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:44:37.712600 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.712576    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "12995527-11dd-40ed-b8af-c686dbd2d6fa" (UID: "12995527-11dd-40ed-b8af-c686dbd2d6fa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:44:37.712702 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.712620    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "12995527-11dd-40ed-b8af-c686dbd2d6fa" (UID: "12995527-11dd-40ed-b8af-c686dbd2d6fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:44:37.712749 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.712707    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12995527-11dd-40ed-b8af-c686dbd2d6fa-kube-api-access-clqls" (OuterVolumeSpecName: "kube-api-access-clqls") pod "12995527-11dd-40ed-b8af-c686dbd2d6fa" (UID: "12995527-11dd-40ed-b8af-c686dbd2d6fa"). InnerVolumeSpecName "kube-api-access-clqls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:37.811581 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.811550    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-maas-api-tls\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:44:37.811581 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.811577    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clqls\" (UniqueName: \"kubernetes.io/projected/12995527-11dd-40ed-b8af-c686dbd2d6fa-kube-api-access-clqls\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:44:37.811581 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.811589    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12995527-11dd-40ed-b8af-c686dbd2d6fa-openshift-service-ca\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:44:37.811823 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:37.811598    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12995527-11dd-40ed-b8af-c686dbd2d6fa-metrics-certs\") on node \"ip-10-0-141-146.ec2.internal\" DevicePath \"\""
Sep 10 21:44:38.308529 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.308424    2583 generic.go:356] "Generic (PLEG): container finished" podID="12995527-11dd-40ed-b8af-c686dbd2d6fa" containerID="99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4" exitCode=0
Sep 10 21:44:38.308529 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.308492    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"
Sep 10 21:44:38.308753 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.308530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q" event={"ID":"12995527-11dd-40ed-b8af-c686dbd2d6fa","Type":"ContainerDied","Data":"99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4"}
Sep 10 21:44:38.308753 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.308613    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q" event={"ID":"12995527-11dd-40ed-b8af-c686dbd2d6fa","Type":"ContainerDied","Data":"f50a43cf5784df81ed9d62a7acb610ce8317f6511fb8df141255458cb579b2e9"}
Sep 10 21:44:38.308753 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.308637    2583 scope.go:117] "RemoveContainer" containerID="99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4"
Sep 10 21:44:38.317403 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.317387    2583 scope.go:117] "RemoveContainer" containerID="99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4"
Sep 10 21:44:38.317736 ip-10-0-141-146 kubenswrapper[2583]: E0910 21:44:38.317718    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4\": container with ID starting with 99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4 not found: ID does not exist" containerID="99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4"
Sep 10 21:44:38.317796 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.317745    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4"} err="failed to get container status \"99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4\": rpc error: code = NotFound desc = could not find container \"99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4\": container with ID starting with 99b6274f4fa0e939518b78971f2be93bb273dbfb90caf9417106689f4a283aa4 not found: ID does not exist"
Sep 10 21:44:38.322481 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.322461    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"]
Sep 10 21:44:38.326660 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:38.326641    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b8f865cbb-c4d7q"]
Sep 10 21:44:39.965003 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:39.964972    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12995527-11dd-40ed-b8af-c686dbd2d6fa" path="/var/lib/kubelet/pods/12995527-11dd-40ed-b8af-c686dbd2d6fa/volumes"
Sep 10 21:44:59.418968 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:44:59.418933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:03.288304 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:03.288270    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77cc466dd6-bngq7_e5e0ce67-7930-4da4-9273-143093bdb094/manager/0.log"
Sep 10 21:45:03.511925 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:03.511880    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b52gc_8fde57b2-669a-442d-8f08-12ea52ef260f/manager/0.log"
Sep 10 21:45:03.742498 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:03.742466    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf_c597fcaf-3cda-4924-b2d5-dfe448498ea8/copy-manifests/0.log"
Sep 10 21:45:03.763557 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:03.763521    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf_c597fcaf-3cda-4924-b2d5-dfe448498ea8/manager/0.log"
Sep 10 21:45:04.309980 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:04.309945    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-84bdd77b86-t7n8p_fbc09c37-a492-49d9-8437-2f13843584fd/maas-api/0.log"
Sep 10 21:45:04.635225 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:04.635109    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-p2fdf_f723b6f4-e96f-45f8-8b0f-251f6483926d/postgres/0.log"
Sep 10 21:45:05.575752 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:05.575707    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-rbhsb_d6be1783-b68d-477a-8778-54832cd79db2/manager/0.log"
Sep 10 21:45:06.049149 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:06.049106    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-52pbp_bd0ad69e-5685-4d6f-a385-876e84cd6a25/manager/0.log"
Sep 10 21:45:06.382932 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:06.382852    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7_7d586b02-0c0b-494c-a319-cf4f777ba63d/istio-proxy/0.log"
Sep 10 21:45:06.502165 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:06.502117    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gb5k_b735a53a-1cf4-48bd-8926-21d406a91b0c/discovery/0.log"
Sep 10 21:45:06.608320 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:06.608283    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-6tj45_bfdba4f5-b266-472d-a4bf-2262c637a203/kube-auth-proxy/0.log"
Sep 10 21:45:06.832673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:06.832638    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dc9wf_ad5f6765-9edd-47fc-a141-d780d3aff1af/istio-proxy/1.log"
Sep 10 21:45:06.946874 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:06.946839    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hz8rz_87bc96bd-059c-43de-82d1-3acbdada0f76/payload-pre-processing/0.log"
Sep 10 21:45:07.070029 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:07.070002    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qzrg6_c273c2b1-fd37-4dcf-a888-17d3a8438bf9/payload-processing/0.log"
Sep 10 21:45:07.180314 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:07.180287    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-647cb99ccb-2bwfw_60ae789f-52b7-4762-9e54-e922b1071dfc/router/0.log"
Sep 10 21:45:07.616035 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:07.615952    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-c249s_1ac8388d-fdbd-4f31-815c-aa514dfeabaa/main/0.log"
Sep 10 21:45:07.953336 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:07.953308    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fz289j_0df849b4-50d7-4311-92c8-68f559f606c8/main/0.log"
Sep 10 21:45:10.091697 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:10.091667    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:45:10.094455 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:10.094433    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8m4m_7b563a9e-843d-4611-b193-7940b3016bcb/ovn-acl-logging/0.log"
Sep 10 21:45:10.362552 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:10.362450    2583 scope.go:117] "RemoveContainer" containerID="a94d6fb77b8c12eda9e3ba7b1b2da2ab4bab0a2dbb92671fd0ae9c51227618cc"
Sep 10 21:45:13.333542 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:13.333487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:13.395179 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:13.395147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:13.704943 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:13.704897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:16.259923 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:16.259890    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jkz74_bfd4681e-1a79-459f-b21c-118f4c0d2646/global-pull-secret-syncer/0.log"
Sep 10 21:45:16.472673 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:16.472642    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log"
Sep 10 21:45:16.691976 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:16.691949    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-146.ec2.internal_c92652107ed38ad17b29af94e128de12/haproxy/0.log"
Sep 10 21:45:19.052790 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:19.052757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:19.119311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:19.119271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g6qq8_656d5e3d-d1a5-4045-b975-fc3390b46564/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:21.496609 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:21.496575    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-rbhsb_d6be1783-b68d-477a-8778-54832cd79db2/manager/0.log"
Sep 10 21:45:21.675935 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:21.675892    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-52pbp_bd0ad69e-5685-4d6f-a385-876e84cd6a25/manager/0.log"
Sep 10 21:45:23.435712 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:23.435684    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-52bth_1c765a24-b9fc-4e49-9275-7be899d0a311/monitoring-plugin/0.log"
Sep 10 21:45:23.469842 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:23.469807    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4d9cs_cae88565-65be-44ad-9572-5008a18e891e/node-exporter/0.log"
Sep 10 21:45:23.494802 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:23.494777    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4d9cs_cae88565-65be-44ad-9572-5008a18e891e/kube-rbac-proxy/0.log"
Sep 10 21:45:23.524450 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:23.524425    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4d9cs_cae88565-65be-44ad-9572-5008a18e891e/init-textfile/0.log"
Sep 10 21:45:25.724800 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:25.724773    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-xcf9l_601f9feb-d42f-4071-a56c-5126b7d4709a/networking-console-plugin/0.log"
Sep 10 21:45:26.515028 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.514991    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"]
Sep 10 21:45:26.515409 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.515394    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12995527-11dd-40ed-b8af-c686dbd2d6fa" containerName="maas-api"
Sep 10 21:45:26.515477 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.515412    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="12995527-11dd-40ed-b8af-c686dbd2d6fa" containerName="maas-api"
Sep 10 21:45:26.515640 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.515618    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="12995527-11dd-40ed-b8af-c686dbd2d6fa" containerName="maas-api"
Sep 10 21:45:26.518570 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.518553    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.520727 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.520707    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vms7j\"/\"kube-root-ca.crt\""
Sep 10 21:45:26.521177 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.521159    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vms7j\"/\"default-dockercfg-jtgj9\""
Sep 10 21:45:26.521257 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.521159    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vms7j\"/\"openshift-service-ca.crt\""
Sep 10 21:45:26.528364 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.528341    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"]
Sep 10 21:45:26.640039 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.640004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-proc\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.640240 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.640050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-sys\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.640240 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.640106    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-podres\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.640240 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.640144    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djfpj\" (UniqueName: \"kubernetes.io/projected/214fb07f-15f4-497e-92e7-ac231a39d84f-kube-api-access-djfpj\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.640240 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.640163    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-lib-modules\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741494 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-proc\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741517    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-sys\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741548    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-podres\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djfpj\" (UniqueName: \"kubernetes.io/projected/214fb07f-15f4-497e-92e7-ac231a39d84f-kube-api-access-djfpj\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741592    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-proc\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741611    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-sys\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-lib-modules\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741695    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-podres\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.741919 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.741714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/214fb07f-15f4-497e-92e7-ac231a39d84f-lib-modules\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.759160 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.759136    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djfpj\" (UniqueName: \"kubernetes.io/projected/214fb07f-15f4-497e-92e7-ac231a39d84f-kube-api-access-djfpj\") pod \"perf-node-gather-daemonset-c9jz8\" (UID: \"214fb07f-15f4-497e-92e7-ac231a39d84f\") " pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:26.829314 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:26.829231    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:27.055711 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:27.055682    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9589c846d-qgkqn_270a1aa7-f318-48ba-afd8-32e2efd47e02/console/0.log"
Sep 10 21:45:27.104857 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:27.104829    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"]
Sep 10 21:45:27.107118 ip-10-0-141-146 kubenswrapper[2583]: W0910 21:45:27.107091    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod214fb07f_15f4_497e_92e7_ac231a39d84f.slice/crio-00e0cb77277382dcf8915429c8feb4ed87217e9ad555744079f108e12808476d WatchSource:0}: Error finding container 00e0cb77277382dcf8915429c8feb4ed87217e9ad555744079f108e12808476d: Status 404 returned error can't find the container with id 00e0cb77277382dcf8915429c8feb4ed87217e9ad555744079f108e12808476d
Sep 10 21:45:27.545116 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:27.545077    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8" event={"ID":"214fb07f-15f4-497e-92e7-ac231a39d84f","Type":"ContainerStarted","Data":"46e374b6f59a1a29fac908b906eeee515072d90130da1d177be881857fec5a6d"}
Sep 10 21:45:27.545311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:27.545136    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8" event={"ID":"214fb07f-15f4-497e-92e7-ac231a39d84f","Type":"ContainerStarted","Data":"00e0cb77277382dcf8915429c8feb4ed87217e9ad555744079f108e12808476d"}
Sep 10 21:45:27.545311 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:27.545164    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:27.561096 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:27.561044    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8" podStartSLOduration=1.56102685 podStartE2EDuration="1.56102685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:45:27.56011702 +0000 UTC m=+2118.087081378" watchObservedRunningTime="2026-09-10 21:45:27.56102685 +0000 UTC m=+2118.087991199"
Sep 10 21:45:28.821055 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:28.821031    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9zgjn_cf5fc50e-6872-4267-8771-2a49a4e91e71/dns/0.log"
Sep 10 21:45:28.850322 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:28.850294    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9zgjn_cf5fc50e-6872-4267-8771-2a49a4e91e71/kube-rbac-proxy/0.log"
Sep 10 21:45:29.045801 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:29.045769    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l7bpv_493c5fe7-fae2-47d0-9f87-55f3702df7ed/dns-node-resolver/0.log"
Sep 10 21:45:29.609520 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:29.609475    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6knd8_437ac578-201e-4cbf-ae6d-607f2eda40a3/node-ca/0.log"
Sep 10 21:45:30.619917 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:30.619884    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7_7d586b02-0c0b-494c-a319-cf4f777ba63d/istio-proxy/0.log"
Sep 10 21:45:31.025357 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:31.025325    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gb5k_b735a53a-1cf4-48bd-8926-21d406a91b0c/discovery/0.log"
Sep 10 21:45:31.051040 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:31.051007    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-6tj45_bfdba4f5-b266-472d-a4bf-2262c637a203/kube-auth-proxy/0.log"
Sep 10 21:45:31.107124 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:31.107102    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dc9wf_ad5f6765-9edd-47fc-a141-d780d3aff1af/istio-proxy/0.log"
Sep 10 21:45:31.122475 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:31.122447    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dc9wf_ad5f6765-9edd-47fc-a141-d780d3aff1af/istio-proxy/1.log"
Sep 10 21:45:31.170170 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:31.170143    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hz8rz_87bc96bd-059c-43de-82d1-3acbdada0f76/payload-pre-processing/0.log"
Sep 10 21:45:31.218288 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:31.218249    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qzrg6_c273c2b1-fd37-4dcf-a888-17d3a8438bf9/payload-processing/0.log"
Sep 10 21:45:31.244480 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:31.244452    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-647cb99ccb-2bwfw_60ae789f-52b7-4762-9e54-e922b1071dfc/router/0.log"
Sep 10 21:45:32.157368 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:32.157336    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x9rvk_44186c88-c46e-49ea-8331-4090b76cb1d3/serve-healthcheck-canary/0.log"
Sep 10 21:45:32.642757 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:32.642726    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-6ph9m_6886a0df-cdef-4730-8483-922187ca8676/insights-operator/0.log"
Sep 10 21:45:32.643385 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:32.643363    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-6ph9m_6886a0df-cdef-4730-8483-922187ca8676/insights-operator/1.log"
Sep 10 21:45:32.671617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:32.671591    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9bs6x_5966eddc-e221-4e73-bcf0-72728ab4a399/kube-rbac-proxy/0.log"
Sep 10 21:45:32.693772 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:32.693747    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9bs6x_5966eddc-e221-4e73-bcf0-72728ab4a399/exporter/0.log"
Sep 10 21:45:32.720042 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:32.720017    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9bs6x_5966eddc-e221-4e73-bcf0-72728ab4a399/extractor/0.log"
Sep 10 21:45:33.558877 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:33.558852    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vms7j/perf-node-gather-daemonset-c9jz8"
Sep 10 21:45:35.191978 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:35.191945    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-77cc466dd6-bngq7_e5e0ce67-7930-4da4-9273-143093bdb094/manager/0.log"
Sep 10 21:45:35.334609 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:35.334558    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b52gc_8fde57b2-669a-442d-8f08-12ea52ef260f/manager/0.log"
Sep 10 21:45:35.498495 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:35.498412    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf_c597fcaf-3cda-4924-b2d5-dfe448498ea8/manager/0.log"
Sep 10 21:45:35.515811 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:35.515786    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd5c8cdd-hvghf_c597fcaf-3cda-4924-b2d5-dfe448498ea8/copy-manifests/0.log"
Sep 10 21:45:36.730693 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:36.730663    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7_7d586b02-0c0b-494c-a319-cf4f777ba63d/istio-proxy/0.log"
Sep 10 21:45:37.142078 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:37.142046    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gb5k_b735a53a-1cf4-48bd-8926-21d406a91b0c/discovery/0.log"
Sep 10 21:45:37.152006 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:37.151981    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-6tj45_bfdba4f5-b266-472d-a4bf-2262c637a203/kube-auth-proxy/0.log"
Sep 10 21:45:37.197384 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:37.197298    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dc9wf_ad5f6765-9edd-47fc-a141-d780d3aff1af/istio-proxy/0.log"
Sep 10 21:45:37.214008 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:37.213977    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dc9wf_ad5f6765-9edd-47fc-a141-d780d3aff1af/istio-proxy/1.log"
Sep 10 21:45:37.250440 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:37.250412    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hz8rz_87bc96bd-059c-43de-82d1-3acbdada0f76/payload-pre-processing/0.log"
Sep 10 21:45:37.288118 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:37.288083    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qzrg6_c273c2b1-fd37-4dcf-a888-17d3a8438bf9/payload-processing/0.log"
Sep 10 21:45:37.306720 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:37.306698    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-647cb99ccb-2bwfw_60ae789f-52b7-4762-9e54-e922b1071dfc/router/0.log"
Sep 10 21:45:38.944886 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:38.944792    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-q2nm7_7d586b02-0c0b-494c-a319-cf4f777ba63d/istio-proxy/0.log"
Sep 10 21:45:39.362837 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:39.362745    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-4gb5k_b735a53a-1cf4-48bd-8926-21d406a91b0c/discovery/0.log"
Sep 10 21:45:39.379176 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:39.379153    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8545c498cb-6tj45_bfdba4f5-b266-472d-a4bf-2262c637a203/kube-auth-proxy/0.log"
Sep 10 21:45:39.417318 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:39.417293    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dc9wf_ad5f6765-9edd-47fc-a141-d780d3aff1af/istio-proxy/0.log"
Sep 10 21:45:39.432252 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:39.432231    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dc9wf_ad5f6765-9edd-47fc-a141-d780d3aff1af/istio-proxy/1.log"
Sep 10 21:45:39.469260 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:39.469232    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hz8rz_87bc96bd-059c-43de-82d1-3acbdada0f76/payload-pre-processing/0.log"
Sep 10 21:45:39.508788 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:39.508763    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-qzrg6_c273c2b1-fd37-4dcf-a888-17d3a8438bf9/payload-processing/0.log"
Sep 10 21:45:39.524403 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:39.524383    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-647cb99ccb-2bwfw_60ae789f-52b7-4762-9e54-e922b1071dfc/router/0.log"
Sep 10 21:45:42.460617 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:42.460575    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-np7rq_f2e5032c-ac57-471c-b0c5-bca7b21236a3/kube-storage-version-migrator-operator/1.log"
Sep 10 21:45:42.462393 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:42.462366    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-np7rq_f2e5032c-ac57-471c-b0c5-bca7b21236a3/kube-storage-version-migrator-operator/0.log"
Sep 10 21:45:43.473233 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:43.473157    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d5mz_577291c3-b7fc-434f-b1a7-ceff5ec93931/kube-multus-additional-cni-plugins/0.log"
Sep 10 21:45:43.495628 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:43.495602    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d5mz_577291c3-b7fc-434f-b1a7-ceff5ec93931/egress-router-binary-copy/0.log"
Sep 10 21:45:43.519349 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:43.519322    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d5mz_577291c3-b7fc-434f-b1a7-ceff5ec93931/cni-plugins/0.log"
Sep 10 21:45:43.540262 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:43.540234    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d5mz_577291c3-b7fc-434f-b1a7-ceff5ec93931/bond-cni-plugin/0.log"
Sep 10 21:45:43.561727 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:43.561705    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d5mz_577291c3-b7fc-434f-b1a7-ceff5ec93931/routeoverride-cni/0.log"
Sep 10 21:45:43.588066 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:43.588044    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d5mz_577291c3-b7fc-434f-b1a7-ceff5ec93931/whereabouts-cni-bincopy/0.log"
Sep 10 21:45:43.609846 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:43.609825    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d5mz_577291c3-b7fc-434f-b1a7-ceff5ec93931/whereabouts-cni/0.log"
Sep 10 21:45:44.047795 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:44.047766    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q9ws7_95ae0a83-f650-4cb9-8e49-3f9ba32f7ce5/kube-multus/0.log"
Sep 10 21:45:44.170982 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:44.170942    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tz9tv_b3857789-0900-400c-b571-2a3980a3c7ee/network-metrics-daemon/0.log"
Sep 10 21:45:44.188047 ip-10-0-141-146 kubenswrapper[2583]: I0910 21:45:44.188020    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tz9tv_b3857789-0900-400c-b571-2a3980a3c7ee/kube-rbac-proxy/0.log"